Journal of Strength and Conditioning Research, 2004, 18(4), 918–920

q 2004 National Strength & Conditioning Association Research Note

DETERMINING THE MAGNITUDE OF TREATMENT

EFFECTS IN STRENGTH TRAINING RESEARCH
THROUGH THE USE OF THE EFFECT SIZE
MATTHEW R. RHEA
Department of Physical Education, Southern Utah University, Cedar City, Utah 84720.
Downloaded from https://journals.lww.com/nsca-jscr by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3lNUiaaDGw22g+tNmeX9juqV+8hbw0iFBcK23WLmeAHWboOp/ZNlNTg== on 08/08/2018

ABSTRACT. Rhea, M.R. Determining the Magnitude of Treat- as well as the significance of the effects. This approach to
ment Effects in Strength Training Research Through the Use of evaluating research findings is especially relevant for re-
Effect Size. J. Strength Cond. Res. 18(4)000–000. 2004.—In order searchers examining strength development. For the pur-
to improve the applicability of research to exercise professionals,
poses of applying strength training research into practice,
it is suggested that researchers analyze and report data in in-
tervention studies that can be interpreted in relation to other the magnitude of the treatment affect may be more im-
studies. The effect size and proposed scale for determining the portant than, or at least just as important as, the repro-
magnitude of the treatment effect can assist strength and con- ducibility of the study. Therefore, the first step to im-
ditioning professionals in interpreting and applying the findings proving the applicability of research to practice is to cal-
of the strength training studies. culate and report statistics that examine the actual mag-
KEY WORDS. statistics, data interpretation, meta-analysis, nitude of a treatment effect.
meaningfulness Various methods have been described for estimating
the magnitude of a treatment effect (4) or an effect size
(ES). Among these, eta squared, omega squared, and Co-
INTRODUCTION hen’s d have been presented as the most common and,
ridging the gap between research and practice generally, the most appropriate. Regardless of the chosen

B is vital to the advancement of both theory and

the application of theory among strength and
conditioning professionals. In fact, this has
been one of the long-standing goals of the Na-
tional Strength and Conditioning Association. A method
calculation for an ES, researchers should report some cal-
culation of the magnitude of a treatment.
In terms of ease of calculation, practicality, and ap-
plication to practice, Cohen’s d and the standardized
mean difference may be the preferred methods of ES cal-
for decreasing this gap is for researchers to analyze and culation in the area of strength and conditioning re-
report their data in a more practical and professionally search. These methods for calculating an ES can easily
applicable manner. Most researchers in exercise science be calculated by the following formulas (1):
report only the statistical probability (the p value) of their
results. This statistic represents the reproducibility of the d 5 (ME 2 MC)/SDC (1)
study. For instance, at the 0.05 level, the results of a par- where ME 5 the mean of the experimental group, MC 5
ticular study can be expected to occur 95 times out of 100. the mean of the control group, and SDC 5 the standard
While the p value is important in determining the deviation of the control group.
amount of confidence we can place in the findings, the
value provides no measure of the magnitude of the treat- Posttest mean 2 Pretest mean
ment effect and is of little value when reported alone. Pre-Post ES 5 (2)
Pretest SD
Probability values are highly affected by sample size
and variance (11). Therefore, this statistic may be mis- As seen above, these ESs represent the difference be-
leading in that large differences in groups or treatments tween two means, divided by the variability among the
may fail to be identified at the 0.05 level simply due to sample. They are therefore reported in standard devia-
small sample sizes or large variance. In contrast, trivial tion units such that an ES of 0.5 represents a difference
differences may reach the 0.05 level if the study had a of ½ of a standard deviation. The ES provides several ben-
large enough sample size. Cohen explained what he con- efits to researchers and professionals. First, it represents
sidered to be ‘‘a fact widely understood among statisti- a standard unit for measuring and interpreting changes
cians’’ (3) that, in the real world, the null hypothesis is in one or more groups that can easily be calculated by
always false and that a large enough sample size will pro- both researchers and strength professionals. Second, it
duce significant results. If a measure of meaningfulness allows for comparisons of different training methods
or magnitude of the treatment effect was reported, re- within a single study. It also provides a method for com-
searchers would be able to evaluate the data more thor- paring treatments in separate but related studies. ESs
oughly and precisely. In fact, if the null hypothesis is al- can be directly compared across studies because they
ways false in the real world, we should be more concerned have been normalized (10). Finally, the ES enables a sin-
about identifying the actual magnitude of a treatment ef- gle study to have a greater impact on theory and practice.
fect. As previous authors (12) have stated, a single study re-
Previous authors (12) have reported the need to con- sulting in a ‘‘yes/no’’ decision at the 0.05 level is unlikely
vince researchers of the importance of reporting an esti- to have an impact on theory or practice. However, includ-
mate of the magnitude of the differences between groups ing the ES offers a valid method for comparisons with

918
 EFFECT SIZES 919

past and future research, and thus has the potential to TABLE 1. Scale for determining the magnitude of effect sizes
contribute greater information to the body of research in strength training research.*
and the practices among professionals. Magni- Recreationally
The calculation of percentage increases in strength tude Untrained trained Highly trained
(the difference between pretest and posttest scores divid-
Trivial ,0.50 ,0.35 ,0.25
ed by the pretest score) is common in an attempt to ac- Small 0.50–1.25 0.35–0.80 0.25–0.50
complish the same goal as the ES: to determine the mag- Moderate 1.25–1.9 0.80–1.50 0.50–1.0
nitude of the changes in strength. However, the calcula- Large .2.0 .1.5 .1.0
tion of percentage increases does not take into consider-
* Untrained 5 individuals who have not been consistently
ation the variance of strength improvements among
trained for 1 year; recreationally trained 5 individuals training
subjects and, therefore, cannot be accurately compared consistently from 1–5 years; highly trained 5 individuals train-
either within or across research studies. By including the ing for as least 5 years.
variance in the calculation, the ES accounts for the var-
iation within and across samples, making it a standard-
ized and more accurate description of the treatment ef-
fect. proposed that a new scale, specific to strength training
The second step to providing more applicable infor- research and the training status of the subjects being
mation for professionals is to determine the relative mag- measured, be used to evaluate the relative magnitude of
nitude of an ES in comparison with other treatment ef- an ES in this area (Table 1). For the purposes of this
fects in strength training research. The development of scale, an untrained individual is considered one who has
an ES scale would provide researchers and professionals not been consistently training for at least 1 year. Recrea-
with benchmarks to which calculated ESs in strength tionally trained populations have been training consis-
training research could be applied in order to determine tently for at least 1 year but less than 5 years. A highly
their relative magnitude. Cohen (1) estimated such a trained individual is one who has been training consis-
scale for behavioral and social sciences. His scale identi- tently for at least 5 years. While numerous definitions of
fied 0.2 as representing a small effect, 0.5 a moderate training status could be argued, a simple characterization
effect, and 0.8 or greater as a large effect. Cohen (2) later such as the one suggested seems most applicable consid-
revisited this scale and proposed that less than 0.41 rep- ering the general sense in which such characterizations
resent a small ES, 0.41–0.70 a moderate ES, and greater will be given.
than 0.70 a large ES. However, Cohen arbitrarily as- With this scale, based on the average ES measured in
signed these magnitudes to be used in the behavioral sci- strength training research as well as the variability
ences, and it is uncertain as to whether Cohen’s scale ac- among such ESs, researchers can now determine the rel-
curately represents magnitudes in strength training re- ative magnitude of an ES calculated from a strength
search. training intervention. When this relative magnitude is re-
With the completion of a number meta-analyses (5–8) ported along with the precise statistical probability, both
as well the calculation and analysis of the magnitude of the researcher and the reader are better able to evaluate
treatment effects in a large number of strength training the overall treatment effect.
sessions, it has become apparent that Cohen’s scale for To illustrate the benefit of calculating and reporting
the social and behavioral sciences does not accurately re- the ES, consider the following example. Readers gain
flect the norm for ESs in strength training research. much more knowledge from a statement such as ‘‘a mod-
Among the nearly 3,000 effect sizes from more than 400 erate effect size was calculated (ES 5 1.5, p 5 0.12)’’ than
studies, including various doses and modes of strength if a researcher simply reported that ‘‘no significant dif-
training, the average ES calculated was about 1.25 (6 ferences were found (p . 0.05).’’ In this case, a moderate
1.0). Cohen (1) stated that a small ES should be one that treatment effect was measured and such an effect could
would not occur by chance and that a large ES should be be expected 88 times out of 100. This information allows
difficult, but possible, to achieve. While it represents only the professional to make a judgment as to whether or not
the average ES in strength training research, an ES of he or she will accept an intervention that can be expected
1.25 is considered very large based on Cohen’s scale for to elicit a moderate treatment effect (compared with other
the social/behavioral sciences. Thus, Cohen’s scale does strength training interventions) 88 times out of 100. If
not accurately reflect the relative magnitudes of treat- only a ‘‘nonsignificant’’ p value is reported, it could only
ment effects in strength training research. be concluded that the program was ineffective. In this
The differences in the size of ESs among social/behav- case, such a conclusion would be incorrect.
ioral research and strength training research is most like-
ly due to the types of treatments employed in behavioral CONCLUSION
sciences and the potential for change among dependent It is imperative that researchers calculate and report
variables. Interventions such as visualization, imagery, some measure of the treatment effect in strength training
and relaxation do not appear to elicit changes in mea- research. With the scale provided it is also possible to
surements that are as large or drastic as the ability of a determine the relative magnitude (small, moderate, or
resistance training program to increase strength mea- large, etc.) of the ESs calculated (primarily Cohen’s d or
sures. These differences in the magnitude of treatment the standardized mean difference) relative to other
effects impede our ability to gain an accurate comparison strength training research. The main goal of strength
to other strength training research when Cohen’s scale is training research should be to determine the magnitude
used. of a treatment effect, rather than solely the reproducibil-
After careful and thorough examination of the ESs ity, of the results of a study. With the effect size statistic
calculated in a variety of strength training research, it is and the scale provided, researchers can provide more
920 RHEA

practical and applicable information to the strength and versus non-periodized strength and power training programs.
conditioning professional. Res. Q. Exerc. Sport. In press.
Readers are directed to the following additional sug- 7. RHEA, M.R., B.A. ALVAR, AND L.N. BURKETT. Single versus
multiple sets for strength: A meta-analysis to address the con-
gested readings: (2, 3, 9, 12). troversy. Res. Q. Exerc. Sport. 73:485–488. 2002.
8. RHEA, M.R., B.A. ALVAR, L.N. BURKETT, AND S.B. BALL. A
REFERENCES meta-analysis to determine the dose-response for strength de-
velopment. Med. Sci. Sports Exerc. 35:456–464. 2003.
1. COHEN, J. Statistical Power Analysis for the Behavioral Sciences
9. ROSNOW, R., R. ROSENTHAL, AND D. RUBIN. Contrasts and cor-
(1st ed.). New York: Academic Press, 1969.
relations in effect-size estimation. Psych. Sci. 11:446–453.
2. COHEN, J. Statistical Power Analysis for the Behavioral Sciences
2000.
(2nd ed.). Hillsdale, NJ: L. Erlbaum Associates, 1988. pp. xxi,
10. SALAZAR, W., S.J. PETRUZZELLO, D.M. LANDERS, J.L. ETNIER,
567.
AND K.A. KUBITZ. Meta-analytic techniques in exercise psy-
3. COHEN, J. Things I have learned (so far). Am. Psych. 45:1304– chology. In: Exercise Psychology. P. Seraganian, ed. New York:
1312. 1990. John Wiley, 1993.
4. FOWLER, R. Point estimates and confidence intervals in mea- 11. THOMAS, J., AND J. NELSON. Research Methods in Physical Ac-
sures of association. Psych. Bull. 98:160–165. 1985. tivity (4th ed.). Champaign, IL: Human Kinetics, 2001.
5. PETERSON, M.D., M.R. RHEA, AND B.A. ALVAR. Maximizing 12. THOMAS, J., W. SALAZAR, AND D.M. LANDERS. What is missing
strength development in athletes: A meta-analysis to deter- in p , .05? Res. Q. Exerc. Sport. 62:344–348. 1991.
mine the dose-response relationship. J. Strength Cond. Res. 1:
11. 2003. Address correspondence to Matthew Rhea, rhea@suu.
6. RHEA, M., AND B. ALDERMAN. A meta-analysis of periodized edu.