Some Cochrane reviews were declared stable or closed, i.e. not in need of updating. For some of t... more Some Cochrane reviews were declared stable or closed, i.e. not in need of updating. For some of them, it has been declared that conclusions will not (or it is unlikely they will) change with further studies. We explored whether there is a discernable decision-making pattern for decisions about the conclusiveness and stabilization of these reviews. We analyzed Cochrane reviews published until April 2020 labeled as stable or closed. We extracted the rationale leading to the decision declaring that the conclusion is not expected to change with further studies. Furthermore, we assessed whether the reviews used GRADE analysis. We extracted data from Summary of findings (SoF) tables on the direction of effect, statistical significance, and I2 values for the first and primary outcomes in SoFs, conclusions in the abstract and review, and implications for practice and future research. We included 40 stable/closed Cochrane reviews. Rationales for their stabilization did not enable any insight into the Cochrane's decision-making algorithm for considering the evidence as conclusive. Among 191 outcomes presented in the SoFs, 70% were rated with either low or very low certainty evidence. None of the reviews mentioned in the text that the review should be stabilized or closed, or that there is sufficient evidence on the subject. Reasons for stabilizing/closing Cochrane reviews were unclear, and we could not discern any pattern of "conclusive review" traits. Definition of systematic review conclusiveness is still lacking, which may contribute to research waste. This article is protected by copyright. All rights reserved.
This study was designed to investigate the pressures and interventions experienced by medical sch... more This study was designed to investigate the pressures and interventions experienced by medical school teachers in relation to grades awarded in assessments of students' knowledge. Of 124 tenured teachers at the University of Split School of Medicine, 91 (73%) participated in the study through the anonymous completion of a questionnaire administered during the 2011-2012 academic year. A total of 42 (46%) teachers reported having experienced some kind of pressure or intervention imposed by students, students' parents or colleagues that was intended to ensure the student obtained a pass or a better grade on an examination. The most common forms of pressure imposed by students were persistent pleading, 'friendly' requests and crying. Students' parents and staff colleagues mostly used 'friendly' requests or indicated that they would like to discuss a student's results. Colleagues who contacted teachers about students' examination results included doctors who did not work at the school and other teachers at the School of Medicine. As a method of preventing the imposition of such pressures and interventions in relation to examination results, teachers suggested that students should be warned against these practices before their courses started and that appropriate policies should be initiated. Some of the teachers rated some specific pressures or interventions as fully or partially acceptable; these included the making of 'friendly' requests, persistent pleading, crying and yelling. Most of the teachers stated that they would report some instances of such interventions to the school management, but 15 (17%) teachers stated that they would not report any type of pressure. It is necessary to raise awareness of the unacceptability of such pressures and interventions in relation to examination results, and to change the current professional culture in which the imposition of such pressures by students, their parents and staff colleagues is considered acceptable and common.
Table S3. Categories of explanations of other bias in analyzed Cochrane risk of bias tables. In t... more Table S3. Categories of explanations of other bias in analyzed Cochrane risk of bias tables. In the 524 analyzed Cochrane reviews that described various sources of other bias, there were 5762 different supporting explanations for judgments of other bias that we categorized into 31 categories. The main text describes the most common categories of explanations, while all the types of explanations is presented in Table S3. (XLSX 37 kb)
Table S2. Cochrane systematic reviews and randomized controlled trials specifically mentioned in ... more Table S2. Cochrane systematic reviews and randomized controlled trials specifically mentioned in the results as those that had different judgment for having no bias, partial information about other bias, or were included in secondary analyses. In 19 Cochrane reviews, all comments that referred to no other bias being identified were judged as unclear. In one review comment, 'no other bias' was judged as both low and high. References to Cochrane reviews for these specific examples are shown in this Additional file. (DOCX 87 kb)
Table S1. Some examples of different versions of support for judgment indicating that no other bi... more Table S1. Some examples of different versions of support for judgment indicating that no other bias was found. In 268 (80%) Cochrane reviews only one version of the comment that no other bias was found was used, while in 69 (20%) reviews Cochrane authors used different expressions in comments to indicate that no other sources of bias were found. Some examples of this varied terminology are shown in Table S1. (DOCX 13 kb)
Table S1. A list of included and excluded studies with a serial number of each record. The supple... more Table S1. A list of included and excluded studies with a serial number of each record. The supplementary table contains a full list of included and excluded studies. Data are arranged in four columns. The first column contains a serial number of each study; second column contains title of a review; third column a remark about whether the review was included in the study or not (yes or no); fourth column describes reason for exclusion if a review was not included. (XLSX 65 kb)
BackgroundAn important part of the systematic review methodology is appraisal of the risk of bias... more BackgroundAn important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews (CSRs) are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in CSRs of interventions published in the Cochrane Database of Systematic Reviews (CDSR).MethodsWe analyzed CSRs published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many CSRs had different judgments for the same supporting explanations.ResultsIn the main analysis we included 10292 judgments and supporting expla...
BackgroundClinical decisions are made based on Cochrane systematic reviews (CSRs), but implementa... more BackgroundClinical decisions are made based on Cochrane systematic reviews (CSRs), but implementation of results of evidence syntheses such as CSRs is problematic if the evidence is not prepared consistently. All systematic reviews should assess risk of bias (RoB) in included studies, and in CSRs this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study we aimed to analyze types and judgments of ‘other bias’ in the RoB tool in CSRs of interventions.MethodsWe analyzed CSRs that included randomized controlled trials (RCTs) and extracted data regarding ‘other bias’ from the RoB table and accompanying support for the judgment. We categorized different types of other bias.ResultsWe analyzed 768 CSRs that included 11369 RCTs. There were 602 (78%) CSRs that had ‘other bias’ domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 CSRs for at least one of the included trials it was indica...
Background and objectiveWe explored how systematic reviews evaluated paracetamol and ibuprofen fo... more Background and objectiveWe explored how systematic reviews evaluated paracetamol and ibuprofen for treating pain in children, as these two non‐opioid analgesics are well‐established medicines included in most national essential medicines lists.Databases and data treatmentWe carried out an overview of systematic reviews (SRs) of randomized controlled trials (RCTs) of interventions (PROSPERO registration: 42016045367). We searched MEDLINE, EMBASE, Cochrane Database of Systematic Reviews (CDSR) and Database of Reviews of Effects (DARE) up to 23 August 2017. We used AMSTAR checklist to analyse methodological quality of included SRs.ResultsWe found 17 SRs with 72 unique RCTs; the majority of those trials included under 100 children. Positive conclusive evidence was found in only one SR, regarding safety of paracetamol. Conclusions of other SRs for efficacy and safety of ibuprofen and paracetamol were inconclusive, unclear, or there was no opinion. Only one SR analysed efficacy of ibuprof...
Some Cochrane reviews were declared stable or closed, i.e. not in need of updating. For some of t... more Some Cochrane reviews were declared stable or closed, i.e. not in need of updating. For some of them, it has been declared that conclusions will not (or it is unlikely they will) change with further studies. We explored whether there is a discernable decision-making pattern for decisions about the conclusiveness and stabilization of these reviews. We analyzed Cochrane reviews published until April 2020 labeled as stable or closed. We extracted the rationale leading to the decision declaring that the conclusion is not expected to change with further studies. Furthermore, we assessed whether the reviews used GRADE analysis. We extracted data from Summary of findings (SoF) tables on the direction of effect, statistical significance, and I2 values for the first and primary outcomes in SoFs, conclusions in the abstract and review, and implications for practice and future research. We included 40 stable/closed Cochrane reviews. Rationales for their stabilization did not enable any insight into the Cochrane's decision-making algorithm for considering the evidence as conclusive. Among 191 outcomes presented in the SoFs, 70% were rated with either low or very low certainty evidence. None of the reviews mentioned in the text that the review should be stabilized or closed, or that there is sufficient evidence on the subject. Reasons for stabilizing/closing Cochrane reviews were unclear, and we could not discern any pattern of "conclusive review" traits. Definition of systematic review conclusiveness is still lacking, which may contribute to research waste. This article is protected by copyright. All rights reserved.
This study was designed to investigate the pressures and interventions experienced by medical sch... more This study was designed to investigate the pressures and interventions experienced by medical school teachers in relation to grades awarded in assessments of students' knowledge. Of 124 tenured teachers at the University of Split School of Medicine, 91 (73%) participated in the study through the anonymous completion of a questionnaire administered during the 2011-2012 academic year. A total of 42 (46%) teachers reported having experienced some kind of pressure or intervention imposed by students, students' parents or colleagues that was intended to ensure the student obtained a pass or a better grade on an examination. The most common forms of pressure imposed by students were persistent pleading, 'friendly' requests and crying. Students' parents and staff colleagues mostly used 'friendly' requests or indicated that they would like to discuss a student's results. Colleagues who contacted teachers about students' examination results included doctors who did not work at the school and other teachers at the School of Medicine. As a method of preventing the imposition of such pressures and interventions in relation to examination results, teachers suggested that students should be warned against these practices before their courses started and that appropriate policies should be initiated. Some of the teachers rated some specific pressures or interventions as fully or partially acceptable; these included the making of 'friendly' requests, persistent pleading, crying and yelling. Most of the teachers stated that they would report some instances of such interventions to the school management, but 15 (17%) teachers stated that they would not report any type of pressure. It is necessary to raise awareness of the unacceptability of such pressures and interventions in relation to examination results, and to change the current professional culture in which the imposition of such pressures by students, their parents and staff colleagues is considered acceptable and common.
Table S3. Categories of explanations of other bias in analyzed Cochrane risk of bias tables. In t... more Table S3. Categories of explanations of other bias in analyzed Cochrane risk of bias tables. In the 524 analyzed Cochrane reviews that described various sources of other bias, there were 5762 different supporting explanations for judgments of other bias that we categorized into 31 categories. The main text describes the most common categories of explanations, while all the types of explanations is presented in Table S3. (XLSX 37 kb)
Table S2. Cochrane systematic reviews and randomized controlled trials specifically mentioned in ... more Table S2. Cochrane systematic reviews and randomized controlled trials specifically mentioned in the results as those that had different judgment for having no bias, partial information about other bias, or were included in secondary analyses. In 19 Cochrane reviews, all comments that referred to no other bias being identified were judged as unclear. In one review comment, 'no other bias' was judged as both low and high. References to Cochrane reviews for these specific examples are shown in this Additional file. (DOCX 87 kb)
Table S1. Some examples of different versions of support for judgment indicating that no other bi... more Table S1. Some examples of different versions of support for judgment indicating that no other bias was found. In 268 (80%) Cochrane reviews only one version of the comment that no other bias was found was used, while in 69 (20%) reviews Cochrane authors used different expressions in comments to indicate that no other sources of bias were found. Some examples of this varied terminology are shown in Table S1. (DOCX 13 kb)
Table S1. A list of included and excluded studies with a serial number of each record. The supple... more Table S1. A list of included and excluded studies with a serial number of each record. The supplementary table contains a full list of included and excluded studies. Data are arranged in four columns. The first column contains a serial number of each study; second column contains title of a review; third column a remark about whether the review was included in the study or not (yes or no); fourth column describes reason for exclusion if a review was not included. (XLSX 65 kb)
BackgroundAn important part of the systematic review methodology is appraisal of the risk of bias... more BackgroundAn important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews (CSRs) are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in CSRs of interventions published in the Cochrane Database of Systematic Reviews (CDSR).MethodsWe analyzed CSRs published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many CSRs had different judgments for the same supporting explanations.ResultsIn the main analysis we included 10292 judgments and supporting expla...
BackgroundClinical decisions are made based on Cochrane systematic reviews (CSRs), but implementa... more BackgroundClinical decisions are made based on Cochrane systematic reviews (CSRs), but implementation of results of evidence syntheses such as CSRs is problematic if the evidence is not prepared consistently. All systematic reviews should assess risk of bias (RoB) in included studies, and in CSRs this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study we aimed to analyze types and judgments of ‘other bias’ in the RoB tool in CSRs of interventions.MethodsWe analyzed CSRs that included randomized controlled trials (RCTs) and extracted data regarding ‘other bias’ from the RoB table and accompanying support for the judgment. We categorized different types of other bias.ResultsWe analyzed 768 CSRs that included 11369 RCTs. There were 602 (78%) CSRs that had ‘other bias’ domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 CSRs for at least one of the included trials it was indica...
Background and objectiveWe explored how systematic reviews evaluated paracetamol and ibuprofen fo... more Background and objectiveWe explored how systematic reviews evaluated paracetamol and ibuprofen for treating pain in children, as these two non‐opioid analgesics are well‐established medicines included in most national essential medicines lists.Databases and data treatmentWe carried out an overview of systematic reviews (SRs) of randomized controlled trials (RCTs) of interventions (PROSPERO registration: 42016045367). We searched MEDLINE, EMBASE, Cochrane Database of Systematic Reviews (CDSR) and Database of Reviews of Effects (DARE) up to 23 August 2017. We used AMSTAR checklist to analyse methodological quality of included SRs.ResultsWe found 17 SRs with 72 unique RCTs; the majority of those trials included under 100 children. Positive conclusive evidence was found in only one SR, regarding safety of paracetamol. Conclusions of other SRs for efficacy and safety of ibuprofen and paracetamol were inconclusive, unclear, or there was no opinion. Only one SR analysed efficacy of ibuprof...
Uploads