Meta-Analysis in Clinical Trials*
Rebecca DerSimonian and Nan Laird
ABSTRACT: This paper examines eight published reviews each reporting results from several
related trials. Each review pools the results from the relevant trials in order to evaluate
the efficacy of a certain treatment for a specified medical condition. These reviews
lack consistent assessment of homogeneity of treatment effect before pooling. We
discuss a random effects approach to combining evidence from a series of experiments
comparing two treatments. This approach incorporates the heterogeneity of effects in
the analysis of the overall treatment efficacy. The model can be extended to include
relevant covariates which would reduce the heterogeneity and allow for more specific
therapeutic recommendations. We suggest a simple noniterative procedure for characterizing the distribution of treatment effects in a series of studies.
KEY WORDS: random effects model, heterogeneity of treatment effects, distribution of treatment effects,
covariate information
INTRODUCTION
Meta-analysis is d e f i n e d h e r e as the statistical analysis of a collection of
analytic results for the p u r p o s e of integrating the findings. Such analyses are
b e c o m i n g increasingly p o p u l a r in medical research w h e r e i n f o r m a t i o n on
efficacy of a t r e a t m e n t is available f r o m a n u m b e r of clinical studies with
similar t r e a t m e n t protocols. If c o n s i d e r e d separately, a n y one s t u d y m a y be
either too small or too limited in scope to c o m e to u n e q u i v o c a b l e or generalizable conclusions a b o u t the effect of treatment. C o m b i n i n g the findings
across such studies r e p r e s e n t s a n attractive alternative to s t r e n g t h e n the evidence a b o u t the t r e a t m e n t efficacy.
The m a i n difficulty in integrating the results f r o m various studies s t e m s
f r o m the s o m e t i m e s diverse nature of the studies, b o t h in t e r m s of design
a n d m e t h o d s e m p l o y e d . S o m e are carefully controlled r a n d o m i z e d experi-
Yale University, School of Medicine, New Haven, Connecticut (R.D.); Harvard University,
School of Public Health, Boston, Massachusetts (N.L.)
*This research was supported by grant CA09424-03 from the National Cancer Institute and
grant GM-29745 from the National Institute of Health. We are grateful to Frederick Mosteller,
Tom Louis, and Katherine Halvorsen for critical readings of various drafts, encouragement, and
advice.
Address reprint request to: RebeccaDerSimonian, Yale University School of Medicine, P.O. Box 3333,
New Haven, CT 06510.
Received March 25, 1986; acceptedApril 7, 1986.
Controlled ClinicalTrials 7:177-188(1986)
© ElsevierSciencePublishing Co., Inc. 1986
52 Vanderbi|t Ave., New York, New York 10017
177
0197-2456/86/$3.50
178
R. DerSimonian and N. Laird
ments while others are less well controlled. Because of differing sample sizes
and patient populations, each study has a different level of sampling error
as well. Thus one problem in combining studies for integrative purposes is
the assignment of weights that reflect the relative "value" of the information
provided in a study. A more difficult issue in combining evidence is that one
may be using incommensurable studies to answer the same question. Armitage [1] emphasizes the need for careful consideration of methods in drawing inferences from heterogeneous but logically related studies. In this setting,
the use of a regression analysis to characterize differences in study outcomes
may be more appropriate [2].
This paper discusses an approach to meta-analysis which addresses these
two problems. In this approach, we assume that there is a distribution of
treatment effects and utilize the observed effects from individual studies to
estimate this distribution. The approach allows for treatment effects to vary
across studies and provides an objective method for weighting that can be
made progressively more general by incorporating study characteristics into
the analysis. We illustrate the use of this model in several examples, and
based on the empirical evidence, suggest a simple noniterative procedure for
testing and estimation.
DATABASE
In a systematic search of the first ten issues published in 1982 of each of
four weekly journals (NEJM, JAMA, BMJ, and Lancet), Halvorsen [3] found
only one article (out of 589) that considered combining results using formal
statistical methods. Our data consist of an ad hoc collection of such articles
from the medical literature found through references provided by colleagues
and through bibliographic references in articles already located [4-11]. The
method we propose applies to several additional articles that have come to
our attention since our original analyses [12-14].
We examine in detail eight review articles each reporting results from
several related trials. Each review pools the results from the relevant trials in
order to evaluate the efficacy of a certain treatment for a specified medical
condition. In most of these reviews the original investigators pool the results
from the relevant trials and estimate an overall treatment effect without first
checking whether the treatment effect across the trials is constant. Others
exclude some trials and combine the results only from trials that are similar
in design and implementation. The investigators who do check for homogeneity of treatment effect before pooling use different criteria to assess this
homogeneity. With two exceptions [6,11], the reviews consider randomized
trials only. The two reviews that include nonrandomized studies analyze the
data from the two groups of studies (randomized and nonrandomized) separately. In this study, we restrict our attention to the results of randomized
trials only. We first describe the eight reviews identifying each by its first
author, and in Table 1 summarize the methods used in each review:
Winship: A review of eight trials that compare the healing rates in duodenal
ulcer patients treated with cimetidine or placebo therapy [4].
~a
¢$}
~-1
Table 1
The M e t h o d s and O u t c o m e Measures Used in the Original Reviews
Outcome measure
Overall effect estimate
Miao
DeSflva
difference in proportions
difference in proportions
difference in proportions
relative risk
Stjernsward
difference in proportions
Baum
difference in proportions
pooled raw data
pooled raw data
weighted average
Mantel-Haenszel
estimate for pooled
relative risk
minimum and
maximum
pooled raw data
Peto
Chalmers
difference in proportions
difference in proportions
unweighted average
Winship
Corm
Test of homogeneity
Chi-Square Test (Qw)
Gilbert, McPeek, and Mosteller
estimate of variance [15]
Mentions lack of heterogeneity
".,.1
~D
180
R. DerSimonian and N. Laird
Conn: A review of nine trials that compare the survival rates in alcoholic
hepatitis patients with steroids or control therapy [5].
Miao: A review of six trials that compare gastric with sham freezing in the
treatment of duodenal ulcer [6]. In addition, this review considers 14 observational and two controlled but nonrandomized studies.
DeSilva: A review of six trials that evaluate the effect of lignocaine on the
incidence of ventricular fibrillation in acute myocardial infarction [7]. This
study originally considered 15 trials but because the trials vary widely in
treatment schedules and doses, some criteria for adequacy of treatment
are established and only six trials that fulfill these requirements are analyzed.
Stjernsward: A review of five trials that compare the 5-year survival rates of
patients with cancer of the breast treated with surgery plus radiotherapy
or surgery alone [8].
Baum: A review of 26 trials that evaluate the efficacy of antibiotics in the
prevention of w o u n d infection following colon surgery [9].
Peto: A review of six trials that evaluate the efficacy of aspirin in the prevention
of secondary mortality in persons recovered from myocardial infarction
[101.
Chalmers: A review of a number of trials that evaluate the efficacy of anticoagulants in the treatment of acute myocardial infarction [11]. Data from
18 surveys employing historical controls (HCT), eight studies employing
alternately assigned controls (ACT), and six randomized controlled trials
(RCT) are given. Three endpoints, total case fatality rates, case fatality rates
excluding early deaths, and thromboembolism rates are considered, although not all studies report all three endpoints. Here we consider thromboembolism and total case fatality rates in the RCTs only. The results from
randomized trials are compared to those of nonrandomized ones (HCTs
and ACTs) in Laird and DerSimonian [16].
METHODS
We consider the problem of combining information from a series of k
comparative clinical trials, where the data from each Hal consist of the number
of patients in treatment and control groups, nT and no, and the proportion
of patients with some event in each of the groups, rT and r o Letting i index
the trials, we assume that the numbers of patients with the event in each of
the study groups are independent binomial random variables with associated
probabilities pTi and Pci, i = 1. . . . . k. The basic idea of the random effects
approach is to parcel out some measure of the observed treatment effect in
each study, say yi, into two additive components: the true treatment effect,
0~, and the sampling error, e~. The variance of ei is the sample variance, s 2,
and is usually calculated from the data of the ith observed sample. The true
treatment effect associated with each trial will be influenced by several factors,
including patient characteristics as well as design and execution of the study.
To explicitly account for the variation in the true effects, the model assumes
Oi = p. + ~i
181
Meta-Analysis in Clinical Trials
where 0i is the true treatment effect in the ith study, Wis the m e a n effect for
a population of possible treatment evaluations, and 8i is the deviation of the
ith s t u d y ' s effect from the population mean. We regard the trials considered
as a sample from this population and use the observed effects to estimate p~
as well as the population variance [var(8) = A2]. Here, ~k2 represents both the
degree to which treatment effects vary across experiments as well as the
degree to which individual studies give biased assessments of treatment effects.
The model just described can thus be characterized by two distinct sampling
stages. First we sample a s t u d y from a population of possible studies with
m e a n treatment effect W a n d variance in treatment effects of A2. Then we
sample observations in the ith s t u d y with underlying treatment effect 0~.
One issue which deserves some attention is the specification of treatment
effect, 0~. Three c o m m o n l y used measures are the risk difference, pr~ - pc/,
the relative risk, pTi/pc,, and the relative odds, [pTi/(1 - pTi)/pcJ(1 - Pc/)].
The relative odds is popular because of its suitability in retrospective or case
control studies, and because it has some interesting mathematical properties.
In particular, if we a s s u m e a constant relative o d d s (0i = ~ or A2 = 0), then
the M a n t e l - H a e n s z e l statistic is optimal for testing Ho: W = 1, and there is
considerable literature on efficient estimates of V- and on m e t h o d s for testing
Ho: 01 = 02 = . . . = Ok. Despite these advantages, the relative odds (and
the closely related relative risk) suffers in interpretability. By far the most
intuitively appealing measure for trials of clinical efficacy is the risk difference,
since it measures actual gains which can be expected in terms of percentages
of patients treated. Besides relevance of the measure and statistical efficiency,
it is also desirable to choose a measure which is nearly constant over studies,
so that the effect of heterogeneity is minimized. Unless there is no treatment
effect at all (pTi = p c / f o r all i), constancy of treatment effect in one scale (say
pzJPci = A for all i) implies variation across studies in another (pT, -- Pc/,
say). Thus it is conceivable that the w r o n g choice of scale could imply heterogeneity in treatment effects which would not exist if a different measure
were used. However, this is not likely to h a p p e n in practice unless there is
a very wide range in the control rates (Pc~) or all the rates are very close to
zero (or one). In such cases, one might w a n t to do the analysis in both the
relative odds and risk difference scales.
HOMOGENEITY OF TREATMENT EFFECT
To evaluate constancy of treatment effect across strata, we use a large
sample test based on the statistic Q = £ wi(yi - yw)2, where yi is the ith
l
treatment effect estimate, ~
= E, wi y ~
wi is the weighted estimator of treat-
m e n t effect, and wi is the inverse of the ith sampling variance. The test statistic
Q is the s u m of squares of the treatment effect about the m e a n where the ith
square is weighted by the reciprocal of the estimated variance. Under the null
hypothesis, Q is approximately a x 2 statistic with k - 1 degrees of freedom;
thus, w h e n each s t u d y has a large sample size relative to the n u m b e r of strata,
Q m a y be used to test Ho: A2 = 0.
182
R. DerSimonian and N. Laird
When y~ is a difference in proportions, rTi -- rc,, we estimate the sampling
variance in the ith study, s2, by
S 2 = rwi (1 -- rTi)/nTi
+ r c i (1 -
(1)
rci)/nci ,
and use Qw = E w;(y~ - ~w)2 to test constancy of treatment effect.
The weights in Q may vary according to the assumptions made about the
sampling variances. For instance, when the sampling variances can be assumed to be equal, then w~, i = 1 . . . . .
k, is the inverse of a common sampling
variance s2. One review [9], which includes a qualitative assessment of homogeneity of treatment effect, uses the method of Gilbert et al. [15] to estimate
the magnitude of the variation across the differences in proportions. Since
the method of Gilbert et al. for estimating the variation in treatment effects
assumes a common sampling variance, we calculate Qu, the analogue of Qw,
assuming equal sampling variances. Here, the treatment effect is again the
difference in proportions, but
W i = S -2,
i = 1. . . . .
k,
where
s~ = ~
s~/k,
i
and s2 is defined in equation (1).
We also use the Q statistic for testing homogeneity in the relative odds
scale. In this scale,
Q L = ~,~ w i ( y i i
yw) 2
where
yi
= In [rTi
Wi ~- S~ 2'
(1
--
rci) /
rci (1
y w "~ ~--a w i y i / ~
i
i
-
rTi)],
Wi,
and
S 2 = [rlTirTi
(1 -- rTi)]-1 + [ncirci (1 - rci)] -1.
In the large sample case, QL is analogous to the goodness-of-fit test in
logistic models [17]. An alternate test statistic for assessing homogeneity is
the likelihood ratio test which is computationally more cumbersome than the
Q statistic used here [18].
ESTIMATION A N D COMPUTATION
Most of the reviews consider the differences in proportions as a measure
of treatment effect (Table 1). For estimating tL and A2 we also restrict our
attention to this scale.
When A2 ~ 0, Qw is used to derive a noniterative estimate of A2 by equating
the sample statistic with the corresponding expected value. This yields a
weighted estimator
183
Meta-Analysis in Clinical Trials
A2 = max {0, {Q,, - (k - 1)} / [ ~ wi - (~_,w~i / ~_, wi)]},
i
i
i
where Qw, ~tw, wi are as described above. The weighted least squares or
Cochran's [19] semiweighted estimator of i• is
(2)
~w = ~ , w* y , / ~ , w*,
i
i
where
w* = ( w i - 1 + 4 2 ) -
I
(3)
The asymptotic standard error of I~w is
s.e. (l~w) = ( ~ w*) -112.
(4)
i
W h e n the sampling variances are a s s u m e d to be equal, these equations reduce
to:
Az, = max [0, {'~P~(y, - y)2 / (k - 1)} - s2],
i
and
s.e. 0~,) = [(s 2 + A2)/k] 1<2,
where
.~ = ~ y i / k
i
and
s 2 = ~ s 2 /k.
i
Rao et al. [20] derive A2 from an u n w e i g h t e d s u m of squares procedure a n d
s h o w that it is also the Minque estimator w h e n the sampling variances are
all equal. The u n w e i g h t e d mean, ~,, is equivalent to the estimate of the
treatment effect in reviews that use the average difference in proportions to
assess the overall treatment efficacy.
With an additional a s s u m p t i o n that yi is N(0;, s2) a n d 0i is N(~, A2), we
also c o m p u t e m a x i m u m likelihood (ML) and restricted m a x i m u m likelihood
(REML) estimates a n d compare t h e m to the noniterative ones. The m a x i m u m
likelihood estimates of the u n k n o w n parameters are those values that maximize the probability density function of the data. In REML estimation, the
likelihood to be maximized is slightly modified to adjust for ~ a n d A2 being
estimated from the same data. The REML estimators are the iterative equivalents of the weighted estimators above. Both ML and REML estimates of
and its s.e. take the form given in equations (2) a n d (4) with weights given
in (3), but differ in the w a y A2 is estimated.
The ML estimating equations are given in Rao et al. [20] and the REML
equations are reviewed by Harville [21]. For implementing the ML or REML
procedures, we use the EM algorithm [22] which is an iterative procedure for
computing m a x i m u m likelihood estimates appropriate w h e n the observations
can be viewed as incomplete data.
184
R. DerSimonian and N. Laird
RESULTS
Homogeneity of Treatment Effect
We present the statistics for testing h o m o g e n e i t y of treatment effect in
Table 2. For these reviews Qw, the weighted statistic in the difference scale,
and QL, the analogous statistic in the log odds scale, imply similar conclusions
about the constancy of treatment effect. The assumption of homogeneity holds
in the reviews by DeSilva [7], Stjernsward [8], Peto [10], and in Chalmers'
[11] randomized controlled trials (case fatality rates). In the remaining five
sets of trials, the evidence suggests heterogeneity of treatment effect irrespective of the scale of measurement.
The review by Peto [10] mentions lack of heterogeneity in treatment effects
across trials. For this review, the Qu statistic supports the homogeneity assumption (p value -- 0.65), whereas both Q~ and QLsupport that assumption
only marginally (p value = 0.12). Baum et al. [9] estimate the variability in
treatment differences using the m e t h o d of Gilbert et al. [15] and conclude
that relative to within study variation (assumed equal for all studies), between
study variation is negligible. This qualitative assessment is not consistent with
the results of Table 2 where a c o m m o n treatment effect across the trials in
this review does not seem to hold in either scale of measurement. The third
review which includes a test of h o m o g e n e i t y before pooling the results [6]
uses a slightly modified version of Q~ to test this hypothesis and the conclusion of lack of h o m o g e n e i t y agrees with the result in Table 2.
As in the review by Peto [10], Qu and Q~ imply different conclusions about
the homogeneity of treatment effect in the review by Winship [4]. In this
review also, the m e t h o d assuming equal weights (Qu) implies homogeneity
of effect while the weighted one implies the opposite.
These results emphasize that the variation in the treatment effect across
several trials is often not negligible and should be incorporated into the anal-
Table 2
Test of H o m o g e n e i t y ~
dff
Winship
Conn
Miao
DeSilva
Stjernsward
Baum
Peto
Chalmers
Thromboembolism
Case fatality rates
QJ
QLe
15.2b
7
8
5
5
4
25
5
21.7b
9.1
2.1
40.4b
9.0
15.6b
20.6b
18.8b
4.4
2.2
35.2b
9.2
5
5
12.3b
3.5
10.3b
2.4
15.6b
aFigures in Tables 2-4 are based on data available at the time of review publication.
bp value <0.10.
CDegrees of freedom.
dQ statistic in difference scale (unequal weights).
~Q statistic in log odds scale (unequal weights).
fQ statistic in difference scale (equal weights).
Qd
7.9
19.3b
20.9b
7.3
2.7
35.3b
3.5
10.6b
1.8
Meta-Analysis in Clinical Trials
185
ysis of the overall treatment efficacy. Lack of homogeneity holds both when
the treatment effect is the difference in proportions and when it is the log
odds. The unweighted statistic which assigns an equal weight to each study
may not be appropriate for testing homogeneity when differences in sample
sizes and/or underlying proportions across studies are large.
Estimation
For all four methods of estimation we present estimates of ~ and its s.e.
in Table 3, and estimates of A2 in Table 4. The estimates of A2, and s.e. (~)
are quite similar in the weighted noniterative method, maximum likelihood,
and restricted maximum likelihood procedures. The A2s from these three
methods are zero or nearly so in the reviews by DeSilva [7], Stjernsward [8],
Peto [10], and Chalmers' [11] randomized trials (case fatality rates). These
same reviews have Q statistics that are small relative to their degrees of
freedom (Table 2). The weighted method and the REML estimation procedures
consistently yield slightly higher values of A2 than the ML procedure. This
is because both these procedures adjust for p, and A2 being estimated from
the same data whereas the ML procedure does not. The estimates of ~ and
its s.e. from these three procedures are expected to be similar since the estimates of A2 are almost equal.
Comparing the unweighted method of moments with the other three methods, we find that the estimates for A2 from this method differ, and sometimes
differ widely, from the estimates of the other three methods but without any
consistent pattern. The estimates of ~ and its s.e. from the unweighted method
also differ from the estimates of the other three methods and these differences
are not necessarily due to the differences in A2s. In Chalmers' [11] randomized
trials (case fatality rates), for instance, even when A2 is zero for all four
methods, the estimate of ~ is 0.042 (s.e. = 0.024) for the unweighted method
while it is 0.029 (s.e. = 0.012) for the other three methods. The original
reviewers report the unweighted average of the observed rate differences
Table 3
Estimated Overall Effects and Their Standard Errors a
Winship
Conn
Miao
DeSilva
Stjernsward
Baurn
Peto
Chalmers
Thromboembolism
Case fatality rates
p,u b
p,wc
p,M a
p,R e
0.406 (0.046)
0.102 (0.092)
0.077 (0.125)
0.026 (0.019)
0.046 (0.020)
0.203 (0.031)
0.018 (0.008)
0.389 (0.058)
0.075(0.072)
0.094(0.111)
0.027(0.019)
0.041 (0.018)
0.208 (0.026)
0.015(0.008)
0.384(0.053)
0.070(0.063)
0.095(0.106)
0.026(0.017)
0.041(0.018)
0.208(0.025)
0.014(0.008)
0.387(0.056)
0.073(0.069)
0.093(0.118)
0.027(0.019)
0.041 (0.018)
0.208(0.026)
0.015(0.008)
0.102 (0.036)
0.042(0.024)
0.079(0.020)
0.029(0.012)
0.078(0.017)
0.029(0.012)
0.078(0.020)
0.029(0.012)
aFigures in parentheses represent the standard errors of the correspondingestimates.
bNoniterative estimates with equal weights.
CNoniterativeestimates with weights to reflect unequal variances.
dMaximumlikelihoodestimates.
eRestricted maximumlikelihoodestimates.
186
R. DerSimonian and N. Laird
Table 4
Estimated Variation in the True Effects
Winship
Conn
Miao
DeSilva
Stjernsward
Baum
Peto
Chalmers
Thromboembolism
Case fatality rates
A u2a
A w2b
/~M2c
A R2d
0.0020
0.0442
0.0716
0.0007
0
0.0072
0
0.0137
0.0208
0.0540
0.0009
0
0.0062
0.0002
0.0096
0.0112
0.0482
0.0006
0
0.0049
0.0002
0.0117
0.0176
0.0638
0.0009
0
0.0057
0.0002
0.0041
0
0.0012
0
0.0007
0
0.0012
0
aNoniterative estimateswith equal weights.
bNoniterative estimates with weights to reflect unequal variances.
CMaximumlikelihoodestimates.
eRestricted maximumlikelihoodestimates.
(0.042) as an estimate of overall treatment efficacy. The weighted estimate of
the treatment effect which weights the observed effects in relation to sample
size is lower than the unweighted average, since some of the larger studies
have smaller estimated treatment effects.
DISCUSSION
We have used a simple random effects model for combining evidence, and
applied it to characterize the distribution of treatment effects in a series of
studies. The model is useful both in summarizing the data and in illustrating
the different kinds of results which one obtains from randomized and nonrandomized studies. In general, studies with greater potential for bias, such
as uncontrolled or nonrandomized ones, show greater treatment effect as well
as greater heterogeneity [2,16].
One important finding that emerges from this investigation is that heterogeneity of treatment effects across studies is common and should be incorporated into the analysis. The random effects model incorporates this heterogeneity, however small, in the analysis of the overall efficacy of the treatment.
The method estimates the magnitude of the heterogeneity, and assigns a
greater variability to the estimate of overall treatment effect to account for
this heterogeneity. In principle, we can extend the model to include pertinent
covariate information [2]. Utilizing covariate information may substantially
reduce the heterogeneity of effects and thus allow for more specific therapeutic
recommendations. This is often difficult in practice, however, since covariate
information may be missing for some studies. Improvement in publication
standards for medical reporting and further methodological work for handling
missing covariate information are needed to strengthen our ability to combine
results from clinical studies.
For estimating the overall treatment effect and the variation of effects across
studies, our results suggest that the weighted noniterative method is an attractive procedure because of the comparability of its estimates with those of
the maximum likelihood methods and because of its relative simplicity. On
Meta-Analysis in Clinical Trials
187
the o t h e r h a n d , the u n w e i g h t e d m e t h o d which ignores differences in sample
sizes yields estimates that often differ from the estimates of the other m e t h o d s .
A p r o b l e m in pooling data w e have not a d d r e s s e d here is that of publication
bias. This p r o b l e m relates to studies being executed, but not reported, usually
because t r e a t m e n t effect has not b e e n found. Reviewers generally r e c o u n t
those studies that a p p e a r to be w o r t h w h i l e and discount those that are unpublished or are not in a g r e e m e n t with a favored g r o u p of studies. The m e t h o d
we describe here r e p r e s e n t s a systematic, quantitative pooling of available
data to resolve controversies about a treatment effect. With each individual
controversy, u n p u b l i s h e d information m a y be elicited a n d along with recent
findings the m e t h o d can be u s e d to u p d a t e the results.
In all our w o r k w e a s s u m e that the sampling variances are k n o w n , although
in reality w e estimate t h e m from the data. Further research n e e d s to be d o n e
in this area as there are alternative estimators that might be preferable to the
ones we use. For instance, if the sample sizes in each s t u d y are small, t h e n
sampling variances based on pooled estimates of the proportions in the treatm e n t a n d control g r o u p s might be better than the ones based on estimates
of p r o p o r t i o n s from the individual studies. A n o t h e r alternative is to shrink
the individual p r o p o r t i o n s towards a p o o l e d estimate before calculating the
variances. Further investigation is n e e d e d before one single m e t h o d emerges
as superior.
REFERENCES
1. Armitage P: Controversies and achievements in clinical trials. Controlled Clin
Trials 5: 67-72, 1984
2. DerSimonian R, Laird N: Evaluating the effect of coaching on SAT Scores: a metaanalysis. Harvard Ed Rev 53: 1-15, 1983
3. Halvorsen K: Combining results from independent investigations: meta-analysis
in medical research. In: Medical Uses of Statistics, Bailar JC, Mosteller F, Eds.
Boston: New England Journal of Medicine (in press)
4. Winship D: Cimetidine in the treatment of duodenal ulcer. Gastroenterology 74:
402-406, 1978
5. Conn H: Steroid treatment of alcoholic hepatitis. Gastroenterology 74: 319-326,
1978
6. Miao L: Gastric freezing: an example of the evaluation of medical therapy by
randomized clinical trials. In: Costs, Risks, and Benefits of Surgery, Bunker JP,
Barnes BA, Mosteller F, Eds. New York: Oxford University Press, 1977, pp. 198-211
7. DeSilva RA, Hennekens CH, Lown B, Casscells W: Lignocaine prophylaxis in
acute myocardial infarction: An evaluation of randomized trials. Lancet ii: 855-858,
1981
8. Stjernsward J: Decreased survival related to irradiation post-operatively in early
operable breast cancer. Lancet ii: 1285-1286, 1974
9. Baum ML, Anish DS, Chalmers TC, Sacks HS, Smith H, Fagerstrom, RM: A survey
of clinical trials of antibiotic prophylaxis in colon surgery: evidence against further
use of no-treatment controls. N Engl J Med 305:795-799, 1981
10. Peto R: Aspirin after myocardial infarction. Lancet i: 1172-1173, 1980 (unsigned
editorial)
11. Chalmers TC, Matta RJ, Smith H, Kunzler AM: Evidence favoring the use of
anticoagulants in the hospital phase of acute myocardial infarction. N Engl J Med
297: 1091-1096, 1977
188
R. DerSimonian and N. Laird
12. Stampfer MJ, Goldhaber SZ, Yusuf S, Peto R, Hennekens CH: Effect of intravenous
streptokinase on acute myocardial infarction: Pooled results from randomized
trials. N Engl J Med 307: 1180-1182, 1982
13. Long-term and short-term beta-blockade after myocardial infarction. Lancet i:
1159-1161, 1982
14. Wortman PM, Yeaton WH: Synthesis of results in controlled trials of coronary
artery bypass graft surgery. Evaluation Studies Review Annual 1983
15. Gilbert JP, McPeek B, Mosteller F: Progress in surgery and anesthesia: benefits
and risks of innovative therapy. In: Costs, Risks, and Benefits of Surgery, Bunker
JP, Barnes BA, Mosteller F, Eds. New York: Oxford University Press, 1977, pp.
124-169
16. Laird N, DerSimonian, R: Issues in combining evidence from several comparative
trials of clinical therapy. In: Proceeding of the XIth International Biometric Conference. 1982, pp. 91-97
17. Breslow NE, Day NE: Statistical methods in cancer research. International Agency
for Research on Cancer, 1980, pp. 136-146
18. Hedges LV, Olkin I: Statistical methods for meta-analysis. London: Academic
Press, 1985, pp. 122-127
19. Cochran WG: Adjustments in analysis. In: Planning and Analysis of Observational
Studies, Moses LE, MosteUer F, Eds. New York: Wiley, 1983, pp. 102-108
20. Rao PS, Kaplan J, Cochran WG: Estimators for the one-way random effects model
with unequal error variances. J Am Stat Assoc 76: 89-97, 1981
21. Harville DA: Maximum likelihood approaches to variance component estimation
and to related problems. J Am Stat Assoc 72: 320-338, 1977
22. Dempster AP, Laird NM, Rubin DB: Maximum likelihood from incomplete data
via the EM algorithm. J R Stat Soc B 39: 1-38, 1977