938

Comments, Opinions, and Reviews

Assessing Bias in Case-Control Studies

Proper Selection of Cases and Controls
Kim Sutton-Tyrrell, DrPH

C ase-control studies are retrospective investi-

gations in which a diseased group (cases)
and a disease-free group (controls) are com-
pared with the aim of uncovering risk factors that
Review of the Odds Ratio
A case-control study is usually evaluated using the
2x2 table, shown in Figure 1. Case and control
groups are considered either "exposed" or "unex-
differ between the groups. Many times, cause-effect posed," depending on whether or not the risk factor
relationships between the risk factor and disease are of interest is present. Using this table, one can
inferred from the results. Many important risk fac- compare the odds of disease in the exposed group
tors have been identified through the use of a case- (A/B) to the odds of disease in the unexposed group
control design, including the relationship of smoking (C/D) to obtain the odds ratio (A/B)/(C/D). The odds
to lung cancer, the use of tampons and toxic shock ratio is most easily calculated using the algebraically
syndrome, and the link between vaginal cancer and equivalent formula (AD)/(BC).3
maternal use of diethylstilbestrol. The odds ratio measures the strength of the asso-
Case-control studies can be executed quickly and ciation between a risk factor (exposure) and disease.
at a relatively low cost, even when the disease of A value of > 1 indicates that the odds of disease are
interest is rare. Such advantages have made the greater when exposed to the specified risk factor. For
case-control design popular, resulting in a progres- example, the interpretation of an odds ratio of 2.0 is
sive increase in its use.1 The validity of a case- that the odds of disease in the exposed group are two
control study, however, is dependent on repre- times the odds of disease in the control group. An
Downloaded from http://ahajournals.org by on October 4, 2020

sentative selection of both the case and control odds ratio of <1 indicates reduced odds of disease
groups.2 Overrepresentation or underrepresentation with exposure to the risk factor, in other words, a
of either of these groups in the study sample results protective effect of the exposure. A value of 1 indi-
in a systematic error referred to as bias. Bias can cates no association between exposure and disease.
cause an inaccurate assessment of the relationship The following examples illustrate how bias in the
between the risk factor and the disease. selection of cases and controls can affect the odds
Although all studies can be affected by bias, case- ratio and, hence, the conclusions of the study.
control studies are particularly susceptible because of
the retrospective nature of the data and the resulting Types of Bias
lack of control the investigator has over many items Referral Bias
of interest. Case-control studies done in a clinical Referral bias occurs when the referral patterns
setting are even further prone to bias because the specific to a community cause an overrepresentation
factors that bring patients to the clinical setting are or underrepresentation of exposed cases in the hos-
often related to the disease or risk factor of interest. pital population as compared to the general popula-
Investigators who embark on case-control studies tion. For referral bias to occur, referral patterns must
must maintain a constant awareness of sources of be related to the exposure of interest. To illustrate
potential bias that could result in invalid conclusions this point, consider the following example, which is
from the study data. This can be a difficult task diagrammed in Figure 2.
because of the numerous and insidious ways that bias In a defined community, patients who have diabe-
can exist.1 The purpose of this paper is to illustrate tes have a 1.9 times greater chance of developing
how bias can result from improper selection of study stroke than patients without diabetes. This commu-
patients. Specific types of bias illustrated are those nity has two major hospitals, A and B. A prominent
particularly applicable to clinically based research. physician who specializes in the treatment and reha-
bilitation of diabetic patients who have suffered
stroke is employed by Hospital A. Aware of the
From the Department of Epidemiology, University of Pitts- reputation of this specialist, physicians in the com-
burgh, Pittsburgh, Pa.
Address for correspondence: Kim Sutton-Tyrrell, DrPH, De- munity refer most of their diabetic patients who
partment of Epidemiology, University of Pittsburgh, 127 Parran manifest symptoms of stroke to Hospital A. Patients
Hall, 130 De Soto Street, Pittsburgh, PA 15261. without a history of diabetes who present with stroke
 Sutton-Tyrrell Bias in Case-Control Studies 939

Cases Controls underrepresentation of exposed cases. A prudent in-

vestigator could pool data from both hospitals to come
up with an unbiased estimate of the relationship be-
A B tween diabetes and stroke.
Exposed Expo—4Cts»m Expotod Control*

Ascertainment Bias
Ascertainment bias occurs when there is inaccu-
Unexposed c D rate ascertainment of either the disease or exposure
Unaxpc—d C a w Uroxpowd Controls
of interest. Case-control studies that rely on chart
review for study data are particularly susceptible to
FIGURE 1. Traditional 2x2 table used in the interpretation this type of bias because the investigator has no
of a case-control study. From this table, the odds ratio is easily control over how the disease and exposure variables
calculated using the formula AD/BC. are ascertained and recorded in the patient chart.
Figure 3 illustrates the following example.
and patients with diabetes who present with other In a general population, if both the presence of a
health problems are referred to Hospitals A and B carotid bruit and the occurrence of transient ischemic
with equal frequency. attack (TLA) are ascertained with perfect accuracy,
The specialist at Hospital A has attracted other the odds of TLA among patients with a bruit are 1.6
physicians and researchers who are interested in times that of patients without a bruit. In Hospital A,
studying stroke in diabetic patients. One investigator interns are required to auscultate for carotid bruits as
initiates a case-control study to assess the impor- part of a standard history and physical performed on
tance of diabetes as a risk factor for stroke. For cases, all admitted patients. Because the history is completed
the investigator chooses all patients who are admit- before the physical, the intern knows the presenting
ted to Hospital A with a diagnosis of first stroke over symptoms of the patient before he or she listens for
a period of 1 year. For controls, the investigator bruits. In patients who present with symptoms of
chooses all patients admitted to Hospital A with a cerebral ischemia, the interns listen very carefully, and
diagnosis other than stroke over the same period. For if a bruit is present, it is correctly diagnosed 90% of
each subject, the investigator identifies whether or the time. In patients with symptoms of ischemia who
not a history of diabetes was present before admis- do not have a bruit, a false positive diagnosis of bruit
sion. The resulting data (Figure 2) lead the investi- is made 15% of the time. The presence of bruit is less
gator to conclude that the odds of developing stroke aggressively sought in patients without symptoms of
Downloaded from http://ahajournals.org by on October 4, 2020

for a patient with diabetes are 3.2 times that for cerebral ischemia, however, and a bruit is correctly
patients without diabetes. diagnosed only 60% of the time. A false positive
If this researcher were to perform an identical study diagnosis of bruit is never made in patients without
at Hospital B, he would be disturbed to find an odds symptoms of cerebral ischemia.
ratio of 0.6, indicating a protective effect of diabetes in An investigator at Hospital A decides to evaluate
the development of stroke. Neither of these conclusions the relationship between carotid bruit and TLA using
is valid because in both cases, the data are biased by the a case-control study. All patients admitted to Hospi-
referral patterns in this community. The sample from tal A with a diagnosis of first TLA over a 3-month
Hospital A contains an overrepresentation of exposed period are selected as cases. A random sample of
cases, whereas the sample from Hospital B contains an patients admitted over the same period with a diag-

Unblased Population
8trok» NoStrofc*

6 22
Odds Ratio - 1.9
NoOtbctM 4 28
FIGURE 2. Referral bias. Referral patterns specific to a
community can cause overrepresentation or underrep-
resentation of exposed cases in a hospitalized popula-
Hospital A Hospital B tion. This type of bias can be avoided by pooling results
8tn*e No Stroke Stroke NoStrok* across hospitals.

5 11 1 11

NoOUMte* 2 14 NottaMm 2 14

Odds Ratio-3.2 Odds Ratio = 0 . 6

 940 Stroke Vol 22, No 7 July 1991

Relationship Between T1A and BnJt cerebral ischemia and are told to report immediately
With Unbiased Ascertainment of Bruft to the hospital if such symptoms occur. Thus, in the
group with carotid bruits, patients experiencing TIA
TW NoTIA are hospitalized almost 100% of the time. Patients
without bruits, however, receive no such warnings,
10 22 and consequently 40% of the time they fail to even
Odds Ratio- 1 . 6
report TLA symptoms to their physician. If symptoms
NoBn* 7 25 are reported, it may be months later, and hospital-
ization never occurs. Ascertainment bias would result
in a hospital-based case-control study because there
% of Each Group Reported to Have BnJt on Hospital Admission would be an underrepresentation of unexposed cases
T1A NoTIA in the sample.
To avoid ascertainment bias, the investigator must
90% 60% make sure that both the exposure and disease of
interest are sought with equal vigor in both the case
15% 0% and control populations. If the investigator does not
have direct control over ascertainment, then select-
ing patients on the basis of the biasing factor can be
done. In the above example, bias in the ascertain-
ment of TIA could be eliminated by selecting for
study only patients without a known carotid bruit
before hospital admission. Both exposed and unex-
Relationship Between H A and Bruft Diaprased
posed cases then would have an equal probability of
on Hospital Admission
having their TLA diagnosed at hospital admission.
TIA No TIA

Berkson Bias
10 13
Odds R a t i o - 3 . 7 Berkson bias, also called admission rate bias, was
first described in 1946.4 The concept underlying this
NoBn* 7 34 bias is that patients with more than one disease or
condition are more likely to be hospitalized than
FIGURE 3. Ascertainment bias. Incomplete ascertainment of patients with only one disease or condition. If a
Downloaded from http://ahajournals.org by on October 4, 2020

the risk factor of interest can result in underrepresentation of case-control study is exploring the relationship be-
exposed patients in the control group. To avoid this type of bias, tween two diseases, this bias can cause an overesti-
the risk factor of interest must be sought with equal vigor in both mation of exposed cases in the hospital population.
the case and control populations. Note: Numbers in the bottom Figure 4 illustrates the following example.
table result after taking into account true positive, false positive, In a denned population of 160 people, 10 people
true negative, and false negative results. Cell A results from nine develop both stroke and some form of cancer, 30
true positives plus one false positive; cell B results from 13 true people develop cancer alone, 30 people develop
positives plus zero false positives; cell C results from six true stroke alone, and 90 people develop neither disease.
negatives plus one false negative; cell D results from 25 true If one were to evaluate the relationship of cancer and
negatives plus nine false negatives. TIA, transient ischemic attack stroke in this population, the odds ratio would be 1.0,
indicating no association. At a given point in time,
the hospitalization rates for each of the four groups
nosis other than stroke or TIA are selected as are as follows: 50% for those with both a history of
controls. Patients with a history of stroke or TIA are stroke and cancer, 10% for those with only one of the
excluded. For all study patients, the standard physi- two diseases, and 5% for those with neither disease.
cal the intern performs at admission is reviewed to A neurologist working in Hospital A notices during
obtain data on the presence or absence of a carotid her rounds that many of her patients who have
bruit. The results of this study indicate that the odds suffered stroke also have a history of some form of
of TLA are 3.7 times higher among patients with a cancer. To determine whether there is a relationship
carotid bruit than among patients without a carotid between stroke and cancer, the neurologist decides
bruit. to conduct a case-control study. As cases, the neu-
This overestimate of the true odds ratio occurred rologist chooses all patients on the medical service
because of bias in the ascertainment of bruit, which who have a history of stroke. As controls, she chooses
resulted in an underrepresentation of exposed pa- all patients on the medical service who do not have a
tients in the control group. A similar spurious in- history of stroke. The charts then are reviewed for a
crease in the odds ratio could occur if there were a history of any form of cancer. The results of the study
bias in the ascertainment of TIA. Consider the (Figure 4) lead the neurologist to conclude that for
following scenario. patients with cancer, the odds of also having a stroke
In this same population, patients with known ca- are 2.8 times higher for patients with cancer than for
rotid bruits receive education about the symptoms of patients without cancer. But this conclusion is invalid
 Sutton-Tyrrell Bias in Case-Control Studies 941

Qeneral Population Unbiased Population

Strofc* hb Stroke (Smoking rate in control population - 33%)
Strok* NoStrofc*
Cmow 10 30
Odds R a t i o - 1 . 0 Snwktr 6 30
90 Odds Ratio - 3 . 0
No Omar 30
Non-Smokw 4 60

Associated HospitaBzatton Rates

Strok* No8tn*«

Cmtxr 50% 10%

NoCmoar 10% 5% Controls - Patients with admission diagnosis of Ml

(Smoking rate in control Population - 60%)
Strofco

Smotur 6 54
O d d s Ratio - 1 . 0
Non-Smoker 4 36
Hoepitaized PopUation
Stroks No Stroke
FIGURE 5. Hospital control bias. If the disease among the
control group is related to the exposure of interest, then an
5 3 overrepresentation of exposed controls can result This type of
Odds R a t t o - 2 . 8 bias can be prevented by selecting patients with a variety of
admission diagnoses for use as controls. MI, myocardial
NsOnoar 3 5
infarction.
FIGURE 4. Berkson bias. Patients with more than one
disease or condition are more likely to be hospitalized than
Downloaded from http://ahajournals.org by on October 4, 2020

interest could be masked. Likewise, if the exposure is

patients with only one condition. This can cause overrepresen-
protective against the disease causing hospitalization
tation of exposed cases in a hospitalized population when the
of the control group, then the relationship could be
exposure of interest is another disease. Berkson bias can be
spuriously increased. Figure 5 illustrates the follow-
avoided by limiting all study subjects to those with a certain
ing example of hospital control bias.
number of major conditions.
Suppose that in the general population the odds of
suffering stroke are three times higher in smokers
because the group of patients with both cancer and than nonsmokers, and the smoking rate among those
stroke (the exposed cases) were overrepresented in who do not develop stroke is 33%. An investigator
the study population. Patients with both diseases working in Hospital A wishes to investigate the
were more likely to be hospitalized and thus chosen relationship between smoking and stroke and decides
for study than patients with only one or neither to do a case-control study. As cases, the investigator
disease. chooses all patients admitted to Hospital A with a
If the investigator had understood Berkson bias, diagnosis of embolic stroke over a 3-month period.
she could have designed her study to avoid this bias As controls, he selects all patients admitted with a
by selecting for study only patients with two major diagnosis of myocardial infarction over the same
health problems. This would cause the hospitaliza- period. The results of this study yield an odds ratio of
tion rates to be approximately equal for the exposed, 1.0, and the investigator concludes that there is no
unexposed, case, and control groups. The investiga- relationship between smoking and embolic stroke.
tor then would have found that cancer occurs just as Invalid results were obtained in this case because
often among patients with other major illnesses as the exposure (smoking) was associated with both the
among those with stroke. disease of interest (embolic stroke) and the disease
causing hospitalization in the control group (myocar-
Hospital Control Bias dial infarction). This caused an overrepresentation of
For hospitalized patients to be an adequate control exposed patients in the control group. The investiga-
group, the disease that resulted in hospitalization tor should have chosen a control group that had a
cannot be related to the exposure of interest. If the smoking rate comparable to the general population.
prevalence of the exposure is higher in the control This error may seem obvious because of the well-
group than in the general population, then a true known association between smoking and myocardial
relationship between the exposure and the disease of infarction. The real problem manifests itself when
 942 Stroke Vol 22, No 7 July 1991

the association between the exposure and the disease the study circumstances. For example, the mecha-
among the control group is unsuspected. One way to nism of subject selection appropriate for one study
guard against this type of bias would be to select a may be inappropriate for another.
control group from hospitalized patients with a vari- Researchers must conduct an evaluation of poten-
ety of different conditions. tial bias during the design phase of a case-control
study. If sources of potential bias are identified, then
Consequences When Bias Goes Unrecognized measures can be undertaken to control or eliminate
In 1929 Raymond Pearl, a noted professor at Johns them. If one waits until the data have been collected,
Hopkins University, published results of a case- there is often little one can do to repair the damage.
control study that revealed a strong negative corre- Although bias is a particular problem with case-
lation between evidence of cancer at autopsy and the control studies,1112 it can affect any type of research.
presence of active tuberculous lesions.5 Cases were Researchers must be trained to understand bias by
patients autopsied in the Johns Hopkins Hospital reviewing examples and applying the concepts to
and found to have a malignant tumor. Control pa- different research scenarios. The alternative is for
tients without malignant tumors on autopsy were researchers to become victims of bias, as in the case of
age-, sex-, and race-matched to the cases. Active Professor Pearl. The Pearl example also underscores
tuberculous lesions were found in 16% of patients the importance of the editorial review process. This
without malignant tumors but in only 7% of patients process must include reviewers who have an under-
with malignant tumors. Pearl thus concluded that standing of how bias can affect study results. Finally,
"there is a definite and marked incompatibility or the investigator or clinician reviewing the literature
antagonism between the two diseases, cancer and may also suffer consequences if the validity of pub-
tuberculosis."5 Researchers at Johns Hopkins were lished research is not continuously scrutinized. Thus,
so impressed by these findings that they began treat- in the design and evaluation of case-control studies as
ing terminal cancer patients with tuberculin and well as all research, an awareness of bias is essential.
published preliminary results in the Lancet.6 Elation
over a cure for cancer was short-lived, however, as it References
was soon discovered that this case-control study 1. Sackett DL: Bias in analytic research. J Chron Dis 1979;32:
suffered from bias that invalidated the results. The 51-63
2. Schlesselman JJ, Stolley PD: Sources of Bias in Case-Control
study sample contained an overrepresentation of Studies. New York, Oxford University Press, Inc, 1982, pp
exposed controls (many control subjects had died 125-143
from tuberculosis), causing a spurious negative cor- 3. Kahn HA, Sempos CT: Relative risk and odds ratio, in
Downloaded from http://ahajournals.org by on October 4, 2020

relation between cancer and tuberculosis. When the Statistical Methods in Epidemiology. New York, Oxford Univer-
study was replicated using a representative control sity Press, Inc, 1989, chap 3
4. Berkson J: Limitations of the application of fourfold tables to
group (patients who died from heart disease), the hospital data. Biometrics Bull 1946;2:47-53
prevalence of tuberculous lesions was found to be the 5. Pearl R: Cancer and tuberculosis. Am J Hyg 1929;9:97-159
same in both cancer and control groups.7 Pearl then 6. Pearl R, Sutton AC, Howard WT Jr: Experimental treatment
was obligated to publish a retraction, which appeared of cancer with tuberculin. Lancet 1929;216:1078-1080
7. Carlson HA, Bell ET: A statistical study of the occurrence of
in Science? Thus, failure to select a proper control cancer and tuberculosis in 11,195 post-mortem examinations./
group and failure to identify the resulting bias led to Cancer Res 1929;13:126-135
a major embarrassment for Pearl, as well as the 8. Pearl R: Discussion: A note on the association of diseases.
journals in which his initial papers were published. Science 1929;70:191-192
Pearl subsequently lost a prestigious appointment to 9. Fee E: Disease and Discovery: A History of the Johns Hopkins
School of Hygiene and Public Health. Baltimore, Johns Hopkins
Harvard, and funding for his Institute of Biological University Press, 1987, pp 136-143
Research was not renewed.9 10. Feinstein AR: Methodologic problems and standards in case-
control research. J Chron Dis 1979;32:35-41
Concluding Comments 11. Cornfield J, Haenszel W: Some aspects of retrospective stud-
ies. / Chron Dis 1960;ll:523-534
Bias can produce spurious associations as well as 12. Mantel N, Haenszel W: Statistical aspects of the analysis of
mask true associations, leading to invalid study con- data from retrospective studies of disease. / Nad Cancer Inst
clusions. The process of recognizing bias can be 1959;22:719-748
difficult and has been described as an intuitive pro-
cess rather than an exact science.10 This is because
measures needed to prevent bias are often specific to KEY WORDS • case-control studies • selection bias