Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Bias in Research: A Review Paper

Download as pdf or txt
Download as pdf or txt
You are on page 1of 7

A Review Paper

Bias in Research
Steven S. Agabegi, MD, and Peter J. Stern, MD

types of bias in such studies include selection bias, perfor-


Abstract mance bias, attrition bias and detection bias.5,6 Other terms
Bias is a systematic inconsistency in research that con- have been used to describe the types of bias (eg, informa-
taminates the primary comparison. There are several tion bias, recall bias, ascertainment bias), but the above 4
forms of bias, and there are specific methods of mini- categories are the most common and will be discussed. In
mizing them in different study designs. The randomized
this article, we will describe the forms of bias and how they
controlled trial (RCT) is the gold standard to which all
other study designs are compared. However, errors can
can be minimized in different study designs.
be made at various stages of a RCT that introduce bias.
Furthermore, not all questions can be addressed by a Types of Studies
RCT, and in some cases another study design may be The advantages and disadvantages of 4 types of study
more appropriate. Observational studies are more prone designs are summarized in Table I.
to bias, but, when properly conducted with rigorous Case reports and case series provide anecdotal infor-
methods to minimize bias, these studies can be valuable mation for rare conditions or new treatments, determine
in clinical research. long-term outcomes of a procedure, and describe the natu-
ral history of a condition or the complication rates after

B
ias is a systematic inconsistency in research stud- a surgical procedure. The size of a case series can range
ies that contaminates a primary comparison and from 2 or 3 to thousands of patients.7
affects the internal validity of the study. Bias is In case series, consecutive cases should be reported so
defined as “any trend in the collection, analysis, that all outcomes, including those that are unfavorable, are
interpretation, publication, or review of data that can lead included. The criteria for case selection should be clearly
to conclusions that are systematically different from the defined with inclusion and exclusion criteria such that
truth.”1 While it cannot be totally eliminated, some study readers can reliably compare their patients with those in
designs are more susceptible than others to particular the case series.7 If a surgical procedure is being studied,
forms of bias, and there are methods of minimizing bias. it should be described in such detail that the reader can
An understanding of the concept of bias and its deleterious replicate that same operation.7 A major drawback of
effects on a study’s validity serves as the building blocks case series is the lack of a comparison group. Historical
for critically appraising the literature. controls (from previous case series) should be used with
Studies are classified according to “levels of evidence” caution because of differences in inclusion and exclusion
on the basis of research design, using internal validity (ie, criteria. Treatment techniques may have improved with
the correctness of the results) as the criterion for hierar- time, and the results may be more a reflection of this
chical rankings. Randomized controlled trials (RCTs) than the actual treatment described.7 Furthermore, the
receive the highest grade, descriptive studies (case series, comparison of case series may be misleading because it
expert opinion) the lowest grade, and observational studies is often unknown whether patient factors and treatments
(cohort and case-control studies) are intermediate.2 Each were similar.8
category is considered methodologically superior to those In case-control studies, patients are selected because
below it,3 primarily because it is less susceptible to bias. they have a certain outcome (eg, a complication, devel-
Studies can be categorized as therapeutic, diagnostic, opment of a disease, mortality). The investigator then
prognostic, and economic analyses. The discussion of bias retrospectively reviews records to identify exposures or
in this article focuses on therapeutic studies, which are risk factors associated with development of that outcome.
the most common type in the orthopedic literature.4 The Case-control studies are always retrospective. For exam-
ple, a case-control design would take a group of patients
with cervical degenerative disc disease (outcome) and
Dr. Agabegi is Resident, and Dr. Stern is Chairman, Department a group without degenerative disc disease to determine
of Orthopaedic Surgery, University of Cincinnati College of
Medicine, Cincinnati, Ohio.
whether a history of whiplash injury (exposure) is associ-
ated with this outcome.
Address correspondence to: Steven S. Agabegi, MD, University In cohort studies, subjects are chosen based on a cer-
of Cincinnati, Department of Orthopaedic Surgery, 231 Albert tain exposure and are followed over time to observe the
Sabin Way, Cincinnati, OH 45267 (tel, 513-558-4592; fax, 513- development of an outcome. This “exposure” may take
558-2220; e-mail, agabegis@yahoo.com).
the form of surgery, any other intervention (therapy, injec-
Am J Orthop. 2008;37(5):242-248. Copyright Quadrant HealthCom tions), injury, patient characteristic (obesity, smoking), or
Inc. 2008. All rights reserved. any other variable that is being investigated. Cohort stud-

242 The American Journal of Orthopedics®


S. S. Agabegi and P. J. Stern

Table I. Study Designs


Study Type Advantages Disadvantages

Case series
Useful for determining long-term outcomes No comparison group
of a given treatment or procedure Often anecdotal
Valuable in describing natural history of a condition, or Prone to many biases
rates of complications after surgical procedure or treatment Difficult to compare with other series
Should not be used to assess treatment efficacy

Cohort study/longitudinal study Good for measuring incidence Time-consuming, expensive


Can study rare exposure No randomization, so selection bias inevitable
Can demonstrate temporal association between exposure Blinding difficult
and outcome (necessary for proving causation) Attrition bias a problem: loss to follow-up
Bias in exposure measurement less likely than common
in case–control studies

Case–control study
Time-efficient, inexpensive Susceptible to confounders and bias (particularly
Effective design for study of rare diseases (may be selection bias, detection bias); both exposure
only feasible method for rare conditions) and disease occurred before study
Best design for diseases with long latent periods Appropriate control group may be difficult to
Can evaluate multiple exposures or risk factors identify
Dependent on reliable, thorough medical
records
Difficult to demonstrate temporal association
between exposure and disease

Randomized controlled trial


Best study for proving causation Expensive, time-consuming
Randomized assignation theoretically distributes both Ethically problematic at times
known and unknown confounders evenly between groups Some surgical questions cannot be
Highest level of evidence when properly conducted addressed with these trials

ies may be prospective or retrospective. Using the above The randomized controlled trial (RCT) is a type of
example, a prospective cohort study may take a cohort of cohort study in which subjects are allocated to either the
patients who have sustained a whiplash injury (exposure) experimental or the control group based on chance. It is
and follow these patients over many years to determine the the only means of controlling for both known and unknown
incidence of cervical degenerative disc disease (outcome). confounding variables (see below). The basic difference
In contrast, a retrospective cohort study may identify a between a RCT and a prospective cohort study is the man-
group of patients who report a history of whiplash injury ner in which subjects are allocated. In a RCT, subject
(either by recall or clinical records) many years ago, and allocation is left to chance, whereas in cohort studies, the
obtain current cervical radiographs of these patients to treatment decision is not random and is determined by the
determine the incidence of degenerative disc disease. The recommendations of the physician as well as the wishes
disadvantage of a retrospective cohort study is the investi- of the patient. Simply stated, patient assignment to either
gators’ reliance on patient records (which are often incom- the experimental or control group is left to chance in the
plete) for exposure measurement.9 As is evident from the RCT, while it reflects personal judgment, decisions, and
examples above, the focus of patient selection in a cohort beliefs (of both physician and patient) in the prospective
study is the presence of exposure. This is in contrast to the cohort study.11 Therefore, selection bias is inherent in the
case-control study in which patients are selected based on design of the cohort study, as will be discussed in more
presence of the outcome. In cohort studies, it is important detail below.
that subjects with similar characteristics be assembled at a When conducted flawlessly under ideal conditions, the
common point in their disease course.10 RCT should theoretically eliminate bias. Practically, how-
As is evident from the above discussion, the method of ever, the RCT is the most difficult to conduct and errors
patient selection in cohort studies and case-control studies can be made at a number of stages that introduce bias and
is reversed. In the former, patients are selected because weaken the validity of the results, as described below.
they have been exposed to a factor or intervention and
are followed to determine whether these patients develop Blinding
a particular outcome. In the latter, patients are selected The term blinding refers to keeping one or more groups
because they have a particular outcome, and the goal is to involved in a research study unaware of the assigned inter-
determine whether these patients had a certain exposure in vention or exposure, so that they will not be influenced by
the past that would be a risk factor for the outcome. that knowledge.12 Traditionally, there have been 3 levels of

May 2008 243


Bias in Research

blinding: single-, double-, and triple-blinding, referring to Confounders can be controlled either in the design of
the blinding of study subjects only; subjects and clinicians the study and/or during the analysis stage such that their
(physician, nurses or anyone evaluating the patient) only; and effect on the results is minimized. It is always preferable to
subjects, clinicians, and those assessing outcome, respec- control for confounders in the design of the study. Control
tively. The higher the level of blinding, the lower the risk of methods include randomization, matching, and restriction
bias. Devereaux and colleagues13,14 showed that physicians (Table II).
and textbooks vary greatly in their interpretations of single, Randomization is the only means of controlling for both
double and triple blinding and recommended abandoning known and unknown (or unrecognized) confounders.17
these terms in favor of a precise description of which groups This is one of the main reasons the RCT is considered
were actually blinded to allocation. These groups include the methodologically superior to other study designs. The
patients, physicians, data collectors, and individuals assessing fundamental criticism of observational studies is that
outcomes, as well as data analysts and authors.13 unrecognized confounders may lead to invalid results.18
The effects of unblinded patients, physicians, and data The random allocation process of the RCT minimizes con-
collectors are discussed below in the section on perfor- founding and therefore selection bias, leading to enhanced
mance bias, and the effects of unblinded investigators internal validity.19
assessing outcomes is discussed under detection bias.
Unblinded data analysts can also introduce bias through Forms of Bias
decisions on patient withdrawals and selection of outcomes
to analyze and report as well as other decisions such as Selection Bias
choice of analytical strategies.13,15,16 Selection bias refers to differences in the characteristics
of subjects included and those excluded for a given study
Confounding Variables or between selected comparison groups of the study.10
A confounding variable is a factor other than the interven- Selection bias is intimately related to the concept of
tion under investigation that obscures the primary com- confounding, but it is a broader term and describes vari-
parison. In clinical research, common confounders include ous sources of differences between the groups. Ideally,
gender, age, socioeconomic status, and comorbidities.10,17 comparison groups should be identical in all respects other
When systematic, confounding is a form of bias and often than the factor under investigation (eg, exposure to a risk
overlaps the forms of bias described below. factor or an investigational treatment), but in reality such
To be a true confounder, a variable must meet 2 criteria: a comparison group does not exist.11 So the goal is to
It must be a risk factor for the outcome of interest and eliminate any element of human intrusion from the process
it must be associated with the explanatory variable.10,17 of patient allocation. Randomization accomplishes this by
Consider, for example, a hypothetical study designed to leaving allocation completely to chance. Any degree of
assess fracture healing after intramedullary nailing of the human intrusion into the randomization of subjects intro-
tibia with radiographic union/nonunion as endpoints in 2 duces selection bias. It is for this reason that the process of
groups of patients, one of which is allowed partial weight- allocation concealment (described below) is critical to the
bearing postoperatively and the other is allowed full integrity of the randomization process.
weight-bearing. Thus, weight-bearing status is the explan- Selection bias is inevitable to some degree in observa-
atory variable under investigation. Two other possible risk tional studies because patients are not randomized. The
factors for nonunion include smoking and age (both meet- challenge, then, is to design observational studies with a
ing the first criterion for being a confounder). For smoking comparison group that is, in all respects, as similar as pos-
and age to be considered confounders, they must also meet sible to the study group, apart from the fact that this group
the second criterion, ie, be associated with the explanatory did not receive the exposure or treatment under investiga-
variable (weight-bearing status). Otherwise, they are likely tion. This can be accomplished by methods that control for
to be equally distributed between comparison groups. Age confounders in the design stage (preferred) as well as in the
is more likely to be associated with weight-bearing status analysis stage (Table II).
(ie, older patients may not be able to comply with weight- The randomization process in RCTs hinges on adequate
bearing restrictions) and therefore be a true confounder. allocation concealment, which reduces selection bias.20
Although smoking may be a risk factor for nonunion (first Allocation concealment, a process distinct from blinding,
criteria), it is no more likely that patients in the full weight- ensures that the random assignment sequence is concealed
bearing group are smokers than patients in the partial from the investigator and patient before and until allocation
weight-bearing group, or vice versa. Therefore, smokers to treatment.21 Blinding, however, refers to the process of
are more likely to be distributed evenly between the 2 concealing the treatment from the patient, investigator, and
groups, minimizing the effect of smoking on the primary other participants (see above) throughout the remainder of
comparison. In this example, age is a confounding variable the study after the patient has been assigned to either the
because it meets both criteria above. Smoking, on the other experimental or control group.
hand, only meets one of the 2 criteria and is unlikely to be To illustrate the importance of allocation concealment,
a confounding variable. consider an investigator enrolling participants in a trial

244 The American Journal of Orthopedics®


S. S. Agabegi and P. J. Stern

Table II. Methods to Control for Confounding Variables


Stage Method Description

Design Randomization Only means of controlling for all confounders (unknown and known) by balancing groups

Matching Goal is equal representation of subjects with certain confounder(s) among comparison groups.
Example: For each group A patient who smokes, have a group B patient who also has this “confounding”
variable. Disadvantage of matching patients on a specific variable (eg, age, smoking) is that it excludes
that variable from analysis. Matching is most commonly done in case–control studies.

Restriction Exclusion of patients with a known confounder from both study groups—only way to totally eliminate
known confounders. Example: Exclude smokers from both groups. Disadvantage: Sample size is reduced.

Analysis Stratification Divides variable into “stratum” and ensures that an equal number of subjects from each stratum is
represented in each group. Example: If age is a confounder, determine “age strata” (20-30, 31-40,
41-50, >51) and ensure that comparison groups have equal numbers from each stratum. Disadvantage is
that sample sizes are reduced; therefore, chances that a treatment effect will be statistically significant are
reduced as well.

Frequency Ensures that proportion of people with the confounder are the same in the 2 groups—with no concern
matching that a specific person in one group is paired with a person in the other group

Statistical Use the confounder as a covariate (eg, regression analysis)


methods

Table III. Summary of Methods to Minimize Bias in Different Study Designsa


Study Design
Type of Bias Randomized Controlled Trial Cohort Case–Control

Selection Randomization Control for confounders by matching Control for confounders by


or restricting matching or restricting

Performance Blinding; avoid contamination Blinding if possible; measurement of exposure Measurement of exposure should
and cointervention should be objective and reliable be objective and reliable; recall bias
often a problem

Attrition Completeness of follow-up; analyze Completeness of follow-up Completeness of follow-up


results on intent-to-treat basis

Detection Blinding of outcome assessor Blinding of outcome assessor Method of information gathering
should be similar for cases and
controls
aAdapted from http://www.cochrane.dk/cochrane/handbook/hbook.26

who has knowledge of the next intervention “assignment.” may be unaware of the ramifications of these decisions and
He may intentionally exclude certain patients based on their effect on the validity of a trial.20
their prognosis because they would have been allocated to There are several methods of allocation concealment
the perceived inappropriate group or, conversely, he may and the method chosen should be clearly described to allow
delay the enrollment of patients until the next “assign- the reader to critically assess a trial. The investigator must
ment” is the perceived appropriate group.20 When those be aware that any mechanism of allocation concealment,
making the decisions about patient eligibility are aware of no matter how meticulous and well-designed, may be
the treatment arm to which patients will be allocated, they potentially vulnerable to being deciphered. Thus, proper
may systematically enroll sicker, or less sick, patients in safeguards must be in place to preserve the integrity of
either the treatment or the control group.22 Intentionally the process. Note that the term randomization is some-
or unintentionally, a researcher may assign subjects to the times used inappropriately in the literature. Any process
treatment arm of a study that may have a different baseline that potentially allows a researcher to figure out which
prognosis than those assigned to the placebo arm. patients received which treatment (eg, alternate allocation
Intentional violation of strict allocation concealment pro- or allocation by date of birth, hospital number, or day of
tocols does occur, sometimes without the knowledge of the week attending clinic) is not a true randomization process
primary investigator. For example, an assistant, resident, because the treatment allocation can be predicted and is
or student who deciphers the allocation scheme and makes not left to chance.23 Such studies are controlled trials rather
decisions on patient assignment based on this knowledge than randomized controlled trials.

May 2008 243


May 2008 245
Bias in Research

Preferred methods of allocation concealment include an unexpected finding, the recording of outcomes, or dif-
remote allocation (in which the individual recruiting the ferential encouragement during performance testing.13,16,28
patient makes a call to a methods center to discover the Unfortunately, many RCTs simply use the terms single-
treatment arm to which the patient is allocated),22 central- blinded or double-blinded without clarifying which group
ized allocation (assignment of subjects performed by a was actually blinded and without explicitly reporting the
separate department or institution, computerized alloca- blinding status of all groups involved in the trial.14
tion, or using coded identical containers/envelopes).24 If An important principle in minimizing performance bias
envelopes are used, they should be sealed and opaque and in RCTs is to ensure that additional treatment other than the
sequentially numbered.25 Invalid methods include alternate intervention be avoided. Additional care (whether invasive
allocation and allocation by date of birth, hospital number, or noninvasive such as counseling or physical therapy) for
or day attending clinic. All of these methods allow the any of the patients becomes a potential confounding factor.
person enrolling subjects to predict which group the sub- Cointervention refers to the provision of additional treat-
ject will be assigned to, and this violates the integrity of ment to either the experimental or control group.29,30 For
randomization. If a study reports that audit checks were example, consider a study aimed at comparing the efficacy
done and revealed no tampering, this increases a reader’s of a bone graft substitute with the efficacy of autogenous
confidence in the integrity of the process.25 iliac crest graft in inducing spinal fusion. If the patients
in whom the bone graft substitute was used also received
Performance Bias electrical bone stimulation postoperatively, this is a coin-
Performance bias is introduced during the treatment or tervention and introduces performance bias. Likewise,
exposure phases of a study and occurs when subjects in contamination refers to provision of the intervention under
comparison groups are systematically given different care investigation in the control group.29,30 For example, using
in ways other than the intervention under investigation. a bone graft substitute in patients randomized to the autog-
To minimize performance bias in randomized controlled enous bone graft group obviously invalidates the results.
trials, the subjects, physicians, and those collecting the To minimize performance bias in observational studies,
data should be “blinded” to the designated intervention the measurement of exposure (to the intervention or fac-
status of each group (Table III 26). Unblinded patients who tor under investigation) should be objective and consistent
know that they are in the control or placebo group may (Table III). For example, in a study to determine risk fac-
be inclined to seek other forms of treatment. Conversely, tors for plantar fasciitis, one factor under investigation may
unblinded patients who know that they are in the treatment be time (number of hours) spent standing during an average
arm of a study may be more likely to report placebo effects day. Proper measurement of this “exposure” (time stand-
or have more favorable expectations. ing) may prove problematic because it hinges on an honest
Likewise, physicians who are not blinded and have and accurate estimate on the part of the subject and may
knowledge of the subject’s group assignment may decide be difficult to quantify with precision. Another risk factor
to withdraw a participant from a study, provide treatments under investigation may be the patient’s weight, which can
other than those under study (cointervention or contamina- be measured more objectively and consistently. Therefore,
tion—see below), and may influence patient compliance or the nature of the exposure or risk factor itself may pose a
reporting of symptoms.13,27 Of course, in surgical trials, it problem because some factors (eg, weight, smoking status,
is impossible to blind the surgeon. It is sometimes reported diabetes) are more objectively measured than others.
in RCTs that surgeons are “blinded” to the randomization In addition, assessing the degree of exposure from
schedule, but this is misleading because this actually repre- personal recall (as done in many case-control studies) is
sents allocation concealment rather than blinding. Use of problematic because recall bias can be introduced.10 For
the term blinding implies that the surgeon was unaware of example, subjects with a positive diagnosis are more likely
the treatment the patient received throughout the course of to recall exposures to risk factors when completing a ques-
the study, which is impossible in a surgical trial. It must be tionnaire than are subjects without the disease or condi-
kept in mind that the purpose of allocation concealment is tion.30 A patient who suffers from the pain of osteoarthritis
to eliminate or minimize selection bias, whereas blinding is more likely to recall prior episodes of injury than an
serves to minimize performance and detection biases (see asymptomatic patient, simply because the former is more
detection bias below).23 So one should distinguish between likely to have ruminated about the possible origin of the
blinding up to the point of patient assignment into either pain. Assessing exposure from medical records may be
the control or experimental groups and blinding that occurs more objective in certain cases, but medical records may be
after patient allocation. The former refers to allocation incomplete, and, often, histories in the medical record are
concealment (which should always be done) while the lat- also obtained by personal recall.
ter represents the proper use of the term blinding (which One advantage of cohort studies over case-control stud-
may not always be possible). ies is that bias in exposure measurement is minimized.
Furthermore, if individuals collecting the data are not For example, with a cohort design, one may select a group
blinded, results may be distorted because of varying inten- of subjects who spend more than 12 hours a day standing
sity of examination, the possibility of repeating a test for at work and another group who stands less than 4 hours
246 The American
244 The American Journal
Journal of
of Orthopedics
Orthopedics®®
S. S. Agabegi and P. J. Stern

a day and follow both groups over time to determine the can show that in a practical “real world” situation, the
incidence of plantar fasciitis in each group. With such a drug actually fails in 95% of patients.
cohort study design, measurement of the exposure is more
accurate because it is the basis of patient selection. With a Detection Bias
case-control design, patients who have already developed Detection bias (also called ascertainment bias and infor-
plantar fasciitis are reviewed to determine whether these mation bias) refers to systematic inconsistency in outcome
patients have a history of prolonged standing. The retrospec- assessment. Subjective measures of outcome (eg, pain,
tive recall of this exposure is difficult to reliably quantify for loose implants on radiographs) are more likely to contribute
reasons mentioned above. Practically, using a cohort design to detection bias than objective measures (eg, negative cul-
to investigate the above problem is time consuming and loss tures, strength measurements). Outcome measures should be
to follow-up is problematic. On the other hand, while a clearly defined, and their measurement should be reliable and
case-control study may be more practical, it is also more sus- standardized for both groups of comparison. For example,
ceptible to performance bias because exposure measurement consider a hypothetical study to determine the effect of the
(time spent standing in a day) can be unreliable. number of daily dressing changes on the healing rate of a
wound. In this study, measurement of the outcome (healing
Attrition Bias rate) may take the form of subjective assessment of healing
Attrition bias relates to patient dropout or exclusion from a (by gross inspection of wound appearance) by 1 observer
study. High dropout rates or systematic differences in the or by objectively measuring the change in diameter of the
number of dropouts between comparison groups introduces wound at weekly intervals by 2 or more independent observ-
bias, because participants who drop out of a study may ers trained to measure wound size in a standardized fashion.
differ systematically from those that remain.31 Ignoring Certainly, the latter is more reliable and reduces the possibility
dropouts in the analysis typically skews the results in favor of bias. Another example of an unreliable outcome measure
of the intervention under investigation. For example, the may be the assessment of spinal fusion on plain radiographs,
results of a study are biased in favor of an experimental which is often inaccurate. A more accurate means of assess-
treatment when patients drop out due to adverse reactions ing fusion may include a CT scan, or if an animal model is
from this treatment, unless these patients are taken into used, histological analysis and biomechanical testing.
account in the analysis.28 Detection bias is minimized in RCTs and cohort studies
The reader should discern whether the investigators if the individuals assessing outcome are blinded to treatment
made every reasonable attempt to follow up with dropouts. allocation, because this ensures that the method of outcome
Unfortunately, there is no recognized dropout rate that is assessment is identical in the comparison groups. Blinding is
considered acceptable, and any designation is likely to be especially important when outcome measures are subjective.24
arbitrary. In general, higher dropout rates are expected (and If those assessing outcome are unblinded, the interpretation
tolerated) in long-term cohort studies because of the nature of marginal findings, or those that require judgment, may be
of the longitudinal design. biased.13,27,28 The observer may be more likely to ascribe a
It is critical that the results of comparative studies be more favorable outcome to the treatment that he prefers.8
analyzed on an intent-to-treat basis. This means that the While the outcome assessor can almost always be
data from patients who withdrew from the study should blinded, there are situations in which this is not possible.
be analyzed along with the data from the patients who Consider a study to determine the time to union of distal
completed the study, regardless of their designation in the radius fractures treated with either surgery or cast immo-
intervention or control arm of the study.22 bilization. If radiographic union is used as the outcome
Failure to apply the intent-to-treat principle threatens measure, it is impossible to blind the individual(s) assess-
the balance that randomization achieves. To illustrate, ing outcome, because the surgical implants are visible on
consider a hypothetical study to determine the efficacy of radiographs. In this instance, if the individual assessing
a new chemotherapeutic medication to treat osteosarco- radiographic union has a preference for surgical treatment,
ma. In a RCT, this medication may be shown to be abso- he may be more likely to describe the fracture treated
lutely effective and safe, resulting in remission in 100% surgically as being healed. Likewise, if tenderness at the
of treated patients when the results are not analyzed on an fracture site is used as the outcome measure of union, the
intent-to-treat basis. However, suppose this medication presence of a surgical incision also precludes blinding of
was so distasteful and caused such severe gastrointestinal the individual assessing outcome, unless steps are taken to
side effects that 95% of patients were noncompliant and hide the incision or if “sham surgery” is performed. When
dropped out of the study prior to completion. In the 5% of the outcome assessor cannot be blinded, the use of more
patients who completed the study, all experienced remis- than one investigator to independently assess outcome
sion. In this situation, excluding the dropouts obviously would help minimize detection bias.
creates a bias because doing so would lead to the errone- In case-control studies, the method of gathering information
ous conclusion that the drug is 100% effective. However, about exposure should be similar between the cases and the
by an intent-to-treat analysis of every patient treated with controls. For example, an investigator may use a questionnaire
this medication (including those that dropped out), one to gather information about exposure from a case but may use

May 2008
May 247
2008 243
Bias in Research

a telephone interview for a control. Furthermore, the search References


1. Last, JM, ed. A Dictionary of Epidemiology. 2nd ed. New York, NY: Oxford
for exposure history may be more intensive for a case than University Press;1988.
for a control subject. To minimize this form of bias, detail 2. Evidence-Based Medicine Working Groups. Evidence-based medicine: a new
approach to teaching the practice of medicine. JAMA. 1992; 268: 2420-2425.
about exposures should be obtained by an investigator who is 3. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational
unaware of whether the subject is a case or a control. studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):
1887-1892.
4. Obremskey WT, Pappas N, Attallah-Wasif E, et al. Level of evidence in orthope-
Conflict of Interest dic journals. J Bone Joint Surg Am. 2005;87(12):2632–2638.
5. Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research.
Funding of research studies from industry sources can create Philadelphia, PA: Saunders;1985: 39-52.
bias due to obvious financial incentives. Studies have shown 6. Greenhalgh T. How to read a paper: assessing the methodological quality of
that research funded by pharmaceutical and medical device published papers. BMJ. 1997;315(7103): 305-308.
7. Carey TS, Boden SD. Critical guide to case series reports. Spine.
industries is more likely to reach a positive conclusion.32-34 2003;28(15):1631-1634.
Recently, Shah and colleagues34 showed that in the spine 8. Hartz A, Marsh JL. Methodological issues in observational studies. Clin Orthop.
2003;413: 33-42.
literature, industry-funded studies were more than 3 times 9. Bhandari M, Guyatt GH, Swiontkowski MF. User’s guide to the orthopaedic
more likely to report a positive result than studies with other literature: how to use an article about prognosis. J Bone Joint Surg Am.
2001;83(10):1555-1564.
funding sources. The authors offered possible explanations 10. Szabo RM. Current concepts review—principles of epidemiology for the ortho-
for this finding, including biases in study design, interpreta- paedic surgeon. J Bone Joint Surg Am. 1998;80(1):111-120.
11. Rochon PA, Gurwitz, JH, Sykora K. Reader’s guide to critical appraisal of cohort
tion, and publication.34 The concern over conflict of interest studies: 1. Role and design. BMJ. 2005;330(7496):895-897.
in and of itself does not invalidate or dispute the findings 12. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommen-
dations for improving the quality of reports of parallel-group randomised trials.
of industry-funded research, which is frequently of high Lancet. 2001;357(9263):1191-1194.
methodological quality. However, the reader should always 13. Devereaux PJ, Manns BJ, Ghali WA. Physician interpretations and textbook
definitions of blinding terminology in randomized controlled trials. JAMA.
note the funding source of a study and perhaps more closely 2001;285(15):2000-2003.
scrutinize the methodology for bias if a conflict of interest 14. Montori VM, Bhandari M, Devereaux PJ. In the dark: the reporting of blinding
status in randomized controlled trials. J Clin Epidemiol. 2002;55(8):787-790.
does indeed exist. 15. Gøtzsche PC. Blinding during data analysis and writing of manuscripts. Control
Clin Trials. 1996;17(4):285-290.
16. Jadad A. Bias in RCTs: beyond the sequence generation. In: Randomised
Summary Controlled Trials. London, England: BMJ Publishing Group;1998:20-36.
Identifying bias in research studies is critical, because 17. Kocher MS, Zurakowski D. Clinical Epidemiology and biostatistics: an orthopae-
dic primer. J Bone Joint Surg Am. 2004;86(3):607-620.
a study that suffers from bias lacks internal validity. 18. Benson K, Hartz AJ. A comparison of observational studies and randomized,
Although bias cannot be totally eliminated from studies, controlled trials. N Engl J Med. 2000;342(25):1878-1886.
19. Grimes DA, Schulz KF. Bias and causal associations in observational research.
the goal is to minimize it. Several pertinent questions can Lancet. 2002;359(9302):248-52.
be asked to identify bias. First, were the groups being com- 20. Schulz KF, Grimes DA. Allocation concealment in randomized trials: defending
against deciphering. Lancet. 2002; 359(9306):614–18.
pared similar in all respects other than the intervention or 21. Schulz KF. Subverting randomization in controlled trials. JAMA.1995;274(18):1456-
exposure of interest? Put simply, are we comparing apples 1458.
22. Bhandari M, Guyatt GH, Swiontkowski MF. User’s guide to the orthopaedic
with apples? Second, were methods used to control for literature: how to use an article about a surgical therapy. J Bone Joint Surg Am.
confounding variables? Although several methods exist to 2001;83(6):916 – 926.
23. Altman DG, Schulz KF, Moher D. The revised CONSORT statement for
control for confounders, randomization is the only method reporting randomized trials: explanation and elaboration. Ann Intern Med.
that theoretically distributes both known and unknown con- 2001;134(8):663-94.
24. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias: dimensions
founders evenly between comparison groups. Furthermore, of methodological quality associated with estimates of treatment effects in con-
were subjects, investigators, and outcome assessors blinded trolled trials. JAMA. 1995;273(5):408-412.
25. Devereaux PJ, McKee MD, Yusuf S. Methodological issues in randomized con-
to the treatment allocation? While it may not be possible trolled trials of surgical interventions. Clin Orthop. 2003;413:25-32.
to blind subjects and investigators, outcome assessors can 26. Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews
of Interventions. Version 5.0.0 [updated February 2008]. The Cochrane
usually be blinded. Finally, was the analysis made on an Collaboration, 2008. Available from www.cochrane-handbook.org.
intent-to-treat basis? 27. Guyatt GH, Sackett DL, Cook DJ; the Evidence-Based Medicine Working Group.
Users’ guides to the medical literature, II: how to use an article about therapy and
The randomized controlled trial represents the highest prevention, A: are the results of the study valid? JAMA.1993;270(21):2598-
level of evidence but it is difficult to conduct and may 2601.
28. Greenhalgh T. Assessing methodological quality. In: How to Read a Paper: The
not be appropriate for some surgical questions. For many Basics of Evidence-Based Medicine. London, England: BMJ Publishing Group;
orthopedic questions, a well-designed observational study 1997:39,62-63.
29. The Canadian Cooperative Study Group. The Canadian trial of aspirin and sulfin-
can be a valuable alternative. Regardless of study design, pyrazone in threatened stroke. N Engl J Med. 1978;299:53-9.
the reader should systematically search for bias and criti- 30. Sackett DL. Bias in analytic research. J Chron Dis. 1979;32(1-2):51-63.
31. Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials.
cally determine the degree to which a study correctly rep- BMJ. 2001;323(7303):42–46.
resents the relationships being assessed. 32. Friedberg M, Saffran B, Stinson TJ, et al. Evaluation of conflict of interest in eco-
nomic analyses of new drugs used in oncology. JAMA.1999;282(15):1453–7.
33. Rochon PA, Gurwitz JH, Simms RW, et al. A study of manufacturer-supported
Authors’ Disclosure Statement trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch
Intern Med. 1994;154(2):157–63.
The authors report no actual or potential conflict of interest in 34. Shah RV, Albert TJ, Bruegel-Sanchez V. Industry support and correlation to
relation to this article. study outcome for papers published in Spine. Spine. 2005;30(9):1099-1104.

This paper will be judged for the Resident Writer’s Award.

248 The American


244 The American Journal
Journal of
of Orthopedics
Orthopedics®®

You might also like