Remedial and Special Education http://rse.sagepub.

com/

Reading Comprehension Instruction for Students With Learning Disabilities, 1995−−2006: A

Meta-Analysis
Sheri Berkeley, Thomas E. Scruggs and Margo A. Mastropieri
Remedial and Special Education 2010 31: 423 originally published online 8 December 2009
DOI: 10.1177/0741932509355988

The online version of this article can be found at:

http://rse.sagepub.com/content/31/6/423

Published by:
Hammill Institute on Disabilities

and

http://www.sagepublications.com

Additional services and information for Remedial and Special Education can be found at:

Email Alerts: http://rse.sagepub.com/cgi/alerts

Subscriptions: http://rse.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

Citations: http://rse.sagepub.com/content/31/6/423.refs.html

>> Version of Record - Nov 5, 2010

OnlineFirst Version of Record - Dec 8, 2009

What is This?

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Article
Remedial and Special Education

Reading Comprehension Instruction 31(6) 423­–436

© 2010 Hammill Institute on Disabilities
Reprints and permission: http://www.
for Students With Learning Disabilities, sagepub.com/journalsPermissions.nav
DOI: 10.1177/0741932509355988

1995–2006: A Meta-Analysis http://rase.sagepub.com

Sheri Berkeley1, Thomas E. Scruggs2, and Margo A. Mastropieri2

Abstract
Meta-analysis procedures were employed to synthesize findings of research for improving reading comprehension of stu-
dents with learning disabilities published in the decade following previous meta-analytic investigations. Forty studies, pub-
lished between 1995 and 2006, were identified and coded. Nearly 2,000 students served as participants. Interventions were
classified as fundamental reading skills instruction, text enhancements, and questioning/strategy instruction—including those
that incorporated peer-mediated instruction and self-regulation. Mean weighted effect sizes were obtained for criterion-
referenced measures: .69 for treatment effects, .69 for maintenance effects, and .75 for generalization effects. For norm-
referenced tests, the mean effect size was .52 for treatment effects. These outcomes were somewhat lower than but
generally consistent with those of previous meta-analyses in their conclusion that reading comprehension interventions
have generally been very effective. Higher outcomes were noted for interventions that were implemented by researchers.
Implications for practice and further research are discussed.

Keywords
reading comprehension, learning disabilities, meta-analysis

A large part of academic learning occurs through reading. & Wei, 2004; Mastropieri et al., 2003; Vaughn, Gersten, &
Reading is not only the ability of a student to accurately and Chard, 2000; Vaughn, Levy, Coleman, & Bos, 2002). For
fluently decode words but also the ability to gain meaning example, Gersten et al. (2001) described the effectiveness of
through the text that he or she reads (Sideridis, Mouzaki, different interventions when they were employed with vari-
Simos, & Protopapas, 2006). Although the goal of the read- ous types of text (e.g., narrative vs. expository) and suggested
ing process is to extract meaning from text, many factors that teaching students to self-regulate their learning was a
can impede a student’s reading comprehension, such as fail- strategy with potential for improving task persistence and
ure to strategically process information and appropriately improving reading comprehension. Kim et al. (2004) particu-
use background knowledge while reading, lack of metacog- larly focused on the effectiveness of graphic organizers in
nitive awareness of learning, knowledge of vocabulary and the context of reading comprehension, whereas Mastropieri
common text structures (i.e., narrative text structure, expos- et al. (2003) specifically focused on reading comprehension
itory text structure), poor reading fluency, and passive instruction for secondary students and paid particular atten-
reading (Gersten, Fuchs, Williams, & Baker, 2001). These tion to peer tutoring, spatial organization, and computer-
challenges are even more significant for students who have assisted instruction. All previous reviews emphasized the
learning disabilities (LD) that further affect their acquisi- overall effectiveness of reading comprehension instruction.
tion of reading comprehension skills (e.g., Mastropieri, In addition to these narrative reviews, a number of compre-
Scruggs, & Graetz, 2003). hensive quantitative syntheses (“meta-analyses”) of research
Interest in reading comprehension instruction increased on reading comprehension instruction for students with LD
substantially after Delores Durkin (1978–1979) wrote about
the need for appropriate reading comprehension instruction. 1
University of Georgia, Athens, GA
Over the past few decades, a substantial number of research 2
George Mason University, Fairfax,VA
studies documenting the efficacy of various strategies for
Corresponding Author:
improving reading comprehension for students with LD have Sheri Berkeley, George Mason University, College of Education and
been published, as have extensive narrative reviews of this Human Development, 110A Krug Hall, MSN 1F2, Fairfax,VA 22030
research (e.g., Gersten et al., 2001; Kim, Vaughn, Wanzek, Email: sberkele@gmu.edu

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

424 Remedial and Special Education 31(6)

have been conducted. Talbott, Lloyd, and Tankersley reading comprehension instruction and, with all effect sizes
(1994) conducted a meta-analysis of 48 relevant research averaged within studies, yielded an overall weighted mean
studies (published between 1978 and 1992), using methods effect size of .72. Mean effect sizes were higher for treat-
described by Glass, McGaw, and Smith (1981), and calcu- ments that employed combinations of direct instruction
lating mean difference effect sizes using the experimental- and strategy instruction and for treatments that employed
control mean difference, divided by the control group criterion-referenced rather than norm-referenced measures.
standard deviation. Differences among categories were Effect sizes larger than .80 have been considered large (e.g.,
evaluated using F ratios. Talbott et al. reported an overall Cohen, 1988), and each of the above meta-analyses yielded
effect size of 1.13 from 255 effect sizes calculated from 48 overall mean effect sizes near to or above this standard, sug-
studies. Of the named interventions, cognitive-behavioral gesting that reading comprehension instruction in general
treatments (including self-questioning, self-monitoring, can have a substantial impact on student learning.
and self-recording) were associated with the descriptively Overall, these three meta-analyses, covering very similar
highest effect sizes, followed in turn by “pre- or mid-reading” publication time periods but employing somewhat different
treatments (e.g., previews or story questions) and cognitive methods, arrived at very similar conclusions: that is, that
interventions (e.g., specific schema or rules, advance orga- reading comprehension instruction was overall very effec-
nizers, outlines). In addition, these authors concluded that tive and that structured cognitive strategies were associated
mean effect sizes were higher for lower level comprehen- with the highest effect sizes. There was also some agreement
sion measures, when researchers delivered the treatment, that researcher-delivered treatments and criterion-referenced,
and when the treatment was compared to a no-treatment rather than norm-referenced, tests were associated with higher
control condition. effect sizes. All three meta-analyses also agreed that length
Mastropieri, Scruggs, Bakken, and Whedon (1996) calcu- of treatment was not observed to be reliably associated with
lated 205 effect sizes from 68 studies (published between mean effect size. Although these three investigations arrived
1976 and 1995), using the procedures recommended by at very similar conclusions, there was some individual dis-
Glass et al. (1981; also see Mastropieri & Scruggs, 1997), agreement; for example, two studies observed no reliable
and reported an overall standardized mean effect size of .98. differences among grade levels, whereas Talbott et al. (1994)
Comparing outcomes descriptively, these authors reported reported higher effect sizes for senior high school students.
that the largest effects were attributed to cognitive strategies In a recent related research synthesis, Gajria, Jitendra,
(similar to the “cognitive-behavioral” category of Talbott Sood, and Sacks (2007) summarized 29 research studies on
et al., 1994), such as summarizing, activating background improving comprehension of expository text by students
knowledge, self-monitoring, and self-questioning. Lower but with LD (reported between 1982 and 2004) and reported
still substantial effects were obtained for studies using text mean unweighted effect sizes of 1.06 for text enhancements,
enhancements (similar to the “cognitive” category of Talbott 1.64 for single comprehension strategies, and 2.11 for mul-
et al., 1994), such as highlighting, illustrations, embedded tiple comprehension strategies. Although these effect sizes
questioning, illustrations, followed finally by skills training were larger than those of previous research synthesis efforts,
and reinforcement (e.g., reinforcement, repeated readings, they are not directly comparable to the previous meta-
vocabulary instruction). Higher effects were also observed analyses because the selection criteria were more restrictive
for criterion-referenced measures (i.e., curriculum or teacher- in some areas (e.g., only studies involving expository text
or researcher-developed comprehension measures) rather were selected) and broader in other areas (e.g., studies on
than norm-referenced tests and for treatments delivered by text comprehension were included that would not have been
researchers rather than teachers (cf. Talbott et al., 1994). classified as “reading comprehension” by standards of the
Swanson and colleagues (Swanson, 1999; Swanson & previous meta-analyses). In addition, this investigation
Hoskyn, 2000; Swanson, Hoskyn, & Lee, 1999) conducted included only a relatively small number of reading compre-
a large-scale meta-analysis of intervention research with stu- hension studies conducted in the past decade. Gajria et al.
dents with LD in a variety of instructional domains, calculat- compared their outcomes descriptively and concluded higher
ing 1,557 effect sizes from 180 studies, removing “outliers,” effect sizes were observed when participants had higher
and weighting each effect size by the reciprocal of the sam- mean IQ scores, when interventions were implemented in
pling variance (Hedges & Olkin, 1985), which had the effect special education classrooms, when materials were specifi-
of somewhat lowering mean effect sizes. Studies were not cally designed for the study, and when researchers were the
included if interventions were not implemented over a primary treatment delivery agents. Therefore, it can be con-
period of 3 days or more. Differences among categories cluded that there has been no comprehensive review exclu-
were evaluated using homogeneity tests (Hedges & Olkin, sively targeting the reading comprehension literature published
1985). Using these criteria, 175 effect sizes from 58 studies in recent years. This in part provides a justification for the
(reported between 1972 and 1997) were directly relevant to present investigation.

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Berkeley et al. 425

The variation in overall obtained mean effect sizes from Practice, Journal of Learning Disabilities, Learning Dis-
previous meta-analyses of reading comprehension, from ability Quarterly, Journal of Special Education, and Reme-
0.72 (Swanson, 1999) to 0.98 (Mastropieri, Scruggs, Bakken, dial and Special Education).
et al., 1996) to 1.13 (Talbott et al., 1994), is no doubt because
of a variety of factors, including differences in criteria and
procedures for study selection; differences in procedures Criteria for Inclusion
for coding, calculating, and analyzing effect sizes; and dif- Studies were included in this meta-analysis if (a) partici-
ferences in procedures for data analysis of obtained out- pants in the study were between kindergarten and Grade 12,
comes. This observed variability underlines the importance (b) the intervention study was primarily designed to improve
of using comparable outcomes (i.e., effect sizes) as a guide- student reading comprehension and specified reading com-
line to, rather than a precise indicator of, overall treatment prehension outcomes, and (c) data were disaggregated for
effectiveness. Cook and Leviton (1980) noted, “While qual- students with disabilities. Studies were included when par-
itative reviews may be equally prone to bias, the descriptive ticipants were specifically identified as having LD (students
accuracy of a point estimate in meta-analysis can have mis- described as reading disabled or dyslexic were considered
chievous consequences because of its apparent ‘objectivity,’ synonymous with LD for the purposes of this synthesis). In
‘precision,’ and ‘scientism’” (p. 455). Keeping this caveat some cases, a small number of the participants were identi-
in mind, meta-analysis can be very informative regarding fied as “at risk” or with other high-incidence disabilities
the value of different interventions and in fact has been very (e.g., Mastropieri et al., 2001); these studies were included
valuable in the past in discriminating effective from inef- when at least two thirds of the participants were identified
fective treatments throughout the history of special educa- as having LD. This was the case in 8 of the 40 studies. In an
tion (Forness, 2001). effort to be comprehensive, studies were included regardless
The purpose of the present research synthesis was to of duration. Finally, studies were included if sufficient data
determine whether reading comprehension instructional were provided to calculate an effect size.
research conducted since the time of the earlier meta-analyses, Studies were excluded if the study was published prior
or more specifically since the Mastropieri, Scruggs, Bakken, to 1995 or was written in a language other than English.
et al. (1996) investigation, has resulted in comparable out- Studies that primarily focused on characteristics of students
comes. We were interested to know whether obtained effect with LD in the area of reading were also not included in this
sizes from more recent research are generally comparable synthesis (e.g., Carr & Thompson, 1996; Jenkins, Fuchs,
to previous research and whether other differences would van de Broek, Espin, & Deno, 2003), nor were single-
be observed, for example, differences in types of treatments. participant studies (e.g., Dowrick, Kim-Rupnow, & Power,
For this reason, intervention categories used a decade ago 2006; Gardill & Jitendra, 1999) or master’s theses (e.g.,
(i.e., Mastropieri, Scruggs, Bakken, et al., 1996) were rep- Heise, 2004) included.
licated in the current meta-analysis.
Coding Instruments, Conventions, and Procedures
Method A coding instrument and coding conventions were devel-
Search Procedures oped for this review, including (a) study identification
information (e.g., first author, publication outlet, year of
The PsycINFO, ERIC, Expanded Academic, Social Sci- publication), (b) characteristics of the sample (e.g., total
ences Citation Index, EBSCO, and Digital Dissertation number of special education students, percentage of male
online databases were used to locate studies using combina- students, mean IQ, achievement description, mean age,
tions of the following keywords: reading disabilities, learn- grade level, school location), (c) description of interven-
ing disabilities, reading comprehension, and strategies. tions (e.g., treatment delivery, teacher type, type of training
Ancestry searches were conducted of both identified stud- passages, time and duration of intervention, use of class-
ies as well as several other literature reviews concerning room peers, use of self-regulation components), (d) design
reading comprehension and students with LD (i.e., Gajria features (e.g., study design, fidelity measure, intervention
et al., 2007; Gersten et al., 2001; Kim et al., 2004; Mastropieri type), and (e) effect sizes (e.g., effect size category and rel-
et al., 2003; Swanson, 1999; Vaughn et al., 2002; Vaughn, evant effect sizes and variances). Coding conditions were
Gersten, et al., 2000). In addition, to ensure a thorough established for each variable. All coders, trained in coding
search, professional journals in the field of special educa- procedures and conventions, regularly met to conduct reli-
tion that commonly publish studies with LD participants ability checking and discuss and resolve coding discrepan-
were hand searched for additional relevant studies (e.g., cies. An initial reliability check, by two independent coders,
Exceptional Children, Learning Disabilities Research and of a randomly selected subset of reports representing 15%

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

426 Remedial and Special Education 31(6)

of the total number of studies yielded an initial reliability size is weighted by the reciprocal of its sampling variance,
coefficient of .92. All discrepancies were resolved to 100% wi = 1/vi (Cooper, 1998; Hedges & Olkin, 1985; for rele-
agreement. vant equations, see Rosenberg et al., 2000, pp. 22–23).
Interventions on reading comprehension were classified This weighted cumulative effect size is reported for all
into categories similar to those employed by Mastropieri, subsequent analyses and presented in tables along with
Scruggs, Bakken, et al. (1996): questioning/strategy instruc- associated 95% confidence intervals (Thompson, 2006).
tion (classified as “self-questioning” in Mastropieri, Scruggs, Furthermore, cross-validation procedures were conducted
Bakken, et al., 1996), text structure, fundamental reading to determine whether these differences were robust across
skills (classified as “basic skills” in Mastropieri, Scruggs, levels of other significant variables or whether observed
Bakken, et al., 1996), and “other.” Studies were categorized differences appeared to systematically vary with levels of
as questioning/strategy instruction if the primary purpose was another variable.
to teach students strategies or involved direct questioning In the present meta-analysis, we employed procedures
of students while reading or if the intervention was designed designed to maximize the validity of our outcomes. Con-
to assist students with becoming independent at self- cerned about the possible compromising effects of combin-
questioning while reading. Interventions were categorized ing different numbers of effect sizes for different studies
as text enhancements if the primary purpose of the interven- (i.e., that some studies would be inappropriately weighted
tion was to supplement or enhance the text to increase com- differently than others), we wished to calculate only one
prehension. Studies were considered to be fundamental effect size per study. However, we were also concerned that
reading skill interventions if they focused on training of basic by combining criterion-referenced (e.g., teacher made)
skills (including, e.g., phonological awareness and/or pho- and norm-referenced measures (where effect sizes from
nics skills) and specifically assessed the impact of this train- norm-referenced measures are typically smaller), we
ing on reading comprehension. Interventions were classified would unfairly advantage studies that employed criterion-
as “other” if they could not be collapsed into one of the other referenced measures. In addition, we anticipated that gen-
categories. eralization and maintenance measures could be expected
Because different types of effect size measures can yield to vary in effectiveness and at any rate address different
very different values, and to preserve the independence of research questions, so we did not wish to combine these
data in the analyses (Hedges & Olkin, 1985), for each study, effects. Therefore, we calculated one effect size each, respec-
one effect size (the effect size that most closely represented tively, for criterion-referenced and norm-referenced mea-
the central purpose of the research) was calculated for the sures and for treatment, maintenance, and generalization
treatment effect for criterion-referenced measures (teacher measures, and we evaluated the effects of each separately.
or researcher developed), and one effect size was calculated Furthermore, rather than combining all possible effects in
for any norm-referenced tests. These categories were not one mean effect size, we selected the outcome most closely
combined in the final analysis. In addition, if studies included related to the research question (e.g., score on story retells
measures of generalization and/or maintenance, one effect rather than consumer satisfaction); and we combined mea-
size was computed for each. Again, these categories were sures when there was more than one single appropriate
not combined in the final analysis. Effect size calculations, effect size (e.g., story retells and comprehension questions).
employing Hedges’s d statistic (Hedges & Olkin, 1985), were Therefore, for a given study, more than one multiple
computed using a statistical software package (MetaWin; effect size may have been calculated, but these were analyzed
Rosenberg, Adams, & Gurevitch, 2000). However, in instan­ separately as (a) criterion-referenced treatment measures,
ces where information needed to be calculated by hand (b) norm-referenced treatment measures, (c) criterion-
(e.g., when deriving from a reported F statistic), at least referenced maintenance measures, (d) norm-referenced
two coders calculated the effect size to ensure accuracy. maintenance measures, or (e) criterion-referenced general-
Where differences were observed for different levels of ization measures (there were no norm-referenced general-
a given variable, homogeneity tests were computed using ization measures identified). For data analysis, we calculated
MetaWin (Rosenberg et al., 2000) to compare mean effect weighted effect sizes with the Hedges d statistic and com-
sizes when the comparison was appropriate and a sufficient pared differences among categories of effects with homoge-
number of cases was present. These tests employed the neity tests (Cooper, 1998; Hedges & Olkin, 1985). These
cumulative effect size, the calculation of which employs techniques are generally thought to lead to more precise
large-sample theory, which states that studies with larger estimates of overall treatment effectiveness (Rosenberg
sample sizes will be associated with lower variances and et al., 2000). When differences were observed, we employed
therefore provide more accurate estimates of the “true” cross-validation procedures to determine whether observed
population effect size. A weighted average was employed differences were robust across levels of other, potentially
to estimate the cumulative effect size, where each effect compromising variables.

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Berkeley et al. 427

Results addition, several studies were unclear about the amount

of time instruction occurred in the comparison condition
Overall Characteristics of the Data Set (n = 7). Peers were used in interventions in 35% of investi-
The final sample consisted of 40 experiments from 40 stud- gations, and self-regulation components were included in
ies published between 1995 and 2006. Studies were located 25% of investigations.
in 16 publication outlets, including doctoral dissertations The most prevalent research design was experimental
(n = 10) and professional journals (n = 30): Journal of designs with random assignment of participants to condi-
Learning Disabilities, Learning Disability Quarterly, tion (52.5%), followed by quasi-experimental designs with
Remedial and Special Education, Journal of Special Edu- matching or nonrandom assignment (40.0%), and pre-post
cation, American Education Research Journal, Learning designs with the pretest as comparison (7.5%). Criterion-
Disabilities Research and Practice, Exceptional Children, referenced tests were employed most frequently as the
Annals of Dyslexia, Brain and Language, Exceptionality, dependent measure (75.0%). Norm-referenced tests were
Learning Disabilities: A Multidisciplinary Journal, Infor- employed in 37.5% of the studies. Maintenance was assessed
mation Technology in Childhood Education Annual, Jour- in 17.5% of the studies using criterion-referenced measures,
nal of Adolescent and Adult Literacy, Elementary School but with only one norm-referenced measure of maintenance.
Journal, and Contemporary Educational Psychology. The Generalization was assessed in 20.0% of studies, in all cases
data set included 1,734 participants with a mean age in with criterion-referenced measures. The majority of research
months of 150.50 (SD = 26.35). The average percentage of comparisons consisted of treatment versus an instructional
male students per study was 67.19% (SD = 11.54). Of the control (65.0%), followed by alternate treatment comparison
40 studies, 25 reported the IQ of students; participants in group (27.5%) and pretest measure (7.5%). Study descrip-
these investigations had a mean IQ of 91.23 (SD = 7.83). tions of instructional control included true controls with no
Achievement scores from a variety of different measures instruction (n = 7), nonreading instruction including study
were reported in 33 investigations (82.5%). These measures skills and vocabulary instruction (n = 2), read-only controls
ranged from formal norm-referenced measures such as with and without accompanying questions and/or repeated
standard scores from the Woodcock Reading Mastery Test reading (n = 6), and “typical” classroom instruction (n = 11).
and grade equivalents on the Gates–MacGinitie Reading Descriptions of “typical” classroom instruction ranged from
Test (n = 25) to criterion-referenced measures such as an very vague (e.g., “typical school based reading program” and
informal reading inventory (n = 3) to more informal reports “reading instruction in normal fashion”) to fairly specific
such as the number of years below grade level (n = 5). (e.g., “systematic reading instruction by regular special edu-
All studies included students with LD, although 8 also cation teacher focusing on decoding and comprehension
included a minority of other students (e.g., attention-deficit/ strategies” and “students reading silently from basal texts,
hyperactivity disorder, remedial reading). followed by teacher-led, large group discussion”).
In all, 15 studies included elementary-age students, 18
included middle school students, 6 included high school
students, and 1 included students in a residential facility Type of Effect Size
for adjudicated youth. Treatments were delivered in large Weighted mean effect sizes (N = 59) were calculated sep-
groups (42.5%), small groups (35.0%), and one-to-one set- arately for treatment, maintenance, and generalization effects
tings (22.5%); one study did not clearly report the treatment and for norm-referenced versus criterion-referenced tests
setting. Interventions were delivered by researchers (40.0%; (because these typically yield effect sizes of different mag-
including authors of the study [28.0%], graduate students nitude). Because research studies frequently reported more
with LD certification and/or experience [10.0%], and a than one of these measures, analyzing these measures sepa-
“trained researcher” [2.0%]), teachers (47.5%; including rately allowed us to evaluate their relative effects without sac-
special education teachers [25.0%], general education rificing independence of individual observations. The overall
teachers [10.0%], both [10.0%], and unspecified [7.5%]), weighted effect size, for criterion-referenced measures, was
technology (7.5%), tutors (2.5%), and parents (2.5%). 0.69 for treatment effects, 0.69 for maintenance effects, and
Interventions lasted a mean of 29.82 sessions (SD = 36.27), 0.75 for generalization effects (M = 0.70 across all criterion-
with a range of 1 to 155. On average, intervention sessions referenced measures). For norm-referenced tests, the mean
were 49.68 min in length, with a minimum of 20 min and a effect size was 0.52 for treatment effects. Only one instance
maximum of 140 min per session. Most studies utilized of a maintenance norm-referenced test (ES = 0.22) and no
comparable amounts of instructional time in intervention instances of a generalization norm-referenced test were
and comparison conditions (n = 22); however, several stud- observed (M = 0.50 across all norm-referenced measures).
ies utilized no comparison, a true control (no instruction), These effect sizes, along with the number of samples and
or differing amounts of instructional time (n = 11). In corresponding confidence intervals, are reported in Table 1.

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

428 Remedial and Special Education 31(6)

Table 1. Hedges’s d by Type of Effect hypermedia (MacArthur & Haynes, 1995), and video vocab-
ulary instruction for text enhancement (Xin & Rieth, 2001).
Type of Effect n ESw 95% CI
Fundamental reading skills. These investigations utilized
packaged intervention programs and had very low student to
Treatment CRT 29 0.69* 0.56 to 0.83 teacher ratios during implementation. The programs included
Maintenance CRT 7 0.69* 0.38 to 1.00 (a) the Behavioral Reading Therapy Program (Burns &
Generalization CRT 8 0.75* 0.47 to 1.03
Treatment NRT 14 0.52* 0.33 to 0.70
Kondrick, 1998), (b) the Failure Free Reading Program
Maintenance NRT 1 0.22 — (Rankhorn, England, Collins, Lockavitch, & Algozzine,
1998), the Auditory Discrimination in Depth Program
Note: n of effect sizes = n of studies; ESw = weighted effect size; (Torgesen et al., 2001), Embedded Phonics (Torgesen et al.,
CI = confidence interval; CRT = criterion-referenced test; NRT = norm-
referenced test. 2001), and (e) the Dyslexia Training Program (Oakland,
*Within-category (Qw) statistical significance, p < .05. Black, Stanford, Nussbaum, & Balise, 1998).
Other interventions. These interventions included a school-
wide cooperative learning program (Stevens & Slavin,
Comparisons of Effect Sizes 1995) and an evaluation of a program with multiple compo-
for Intervention Treatments nents (Project Read; Laub, 1997).
Outcomes. Type of measure, mean weighted effect sizes,
Across all effect types, 27 studies were categorized as and frequencies and confidence intervals associated with
questioning/strategy instruction (67.5%), 6 studies were these interventions are reported in Table 2. For criterion-
categorized as text enhancements (15.0%), 5 studies were referenced measures, overall, the two interventions with the
categorized as fundamental reading skills training (12.5%), largest number of effect sizes—questioning/strategy instruc-
and 2 studies were categorized as “other” (5.0%). Associ- tion and text enhancements—were associated with mean
ated weighted mean effect sizes, along with the number weighted effect sizes that were moderate to large in magni-
of samples and corresponding confidence intervals, are tude. The effect for fundamental reading skills was large,
reported in Table 2. and a small effect was noted for “other”; however, each of
Questioning/strategy instruction. Interventions in this cate- these latter outcomes was associated with only a single
gory included direct questioning of students while reading, effect size and was therefore not included in the analysis.
teaching students comprehension strategies (including ques- Overall, outcomes by intervention type were not statistically
tioning strategies), and assisting students with becoming different according to a homogeneity test, Q(1, N = 28) = 0.60,
independent at self-questioning while reading. Some exam- p = .71. Significant heterogeneity was found within each of
ples of teacher-directed questioning include elaborative the questioning/strategy instruction and text enhancements
interrogation where, in this case, students read passages categories, suggesting considerable variability was associ-
about animals and were trained to ask themselves why cer- ated with each of these treatments.
tain facts (e.g., that camels have long eyelashes) may be true On norm-referenced tests, mean outcomes were also sim-
(Mastropieri, Scruggs, Hamilton, et al., 1996); a guided ilar, although lower overall as expected, and paralleled effect
reading approach for identification of story themes (Williams, sizes from criterion-referenced measures, as shown in Table 2.
1998); main idea strategy instruction with self-monitoring Descriptively, highest effect sizes were associated with fun-
training where students assessed their ability to apply the damental reading skills instruction, followed by questioning/
main idea strategy (Jitendra, Hoppes, & Xin, 2000); peer- strategy instruction, and text enhancements. These differ-
assisted learning strategies where peers question each other ences, however, were not statistically significant according
on the main topic, relevant information, and text summary to a homogeneity test, Q(2, N = 14) = 2.71, p = .40. Nei-
(e.g., Saenz, Fuchs, & Fuchs, 2005); and text structure anal- ther were significant differences observed within any of the
ysis, where students were trained to identify the type of text treatment categories. Outcomes on criterion-referenced mea-
structure in paragraphs (main idea, with a summary idea and sures for generalization (M = 0.75) and maintenance effects
supporting statements; list, with information presented in a (M = 0.69) were associated with too few cases of text
list format; and order, with listed information presented enhancement or fundamental reading skills treatments for
in a specific order) to facilitate comprehension (Bakken, statistical comparisons to be made. However, it was inter-
Mastropieri, & Scruggs, 1997). esting to note that mean effect sizes for these measures were
Text enhancements. Examples of interventions in this very similar to treatment effects.
category included in-text question placement with and with- Extreme cases. Extreme cases (studies very high or very
out feedback (Peverly & Wood, 2001), graphic organizers low effect sizes) were closely examined in an attempt to
designed for both narrative (Boyle, 1996) and expository explain the variability within specific categories. In the
text (DiCecco & Gleason, 2002), technology including questioning/strategy instruction category, five studies were

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Berkeley et al. 429

Table 2. Hedges’s d (Treatment Effects) by Type of Intervention and Treatment Delivery

Criterion-Referenced Tests Norm-Referenced Tests

n ESw 95% CI n ESw 95% CI

Type of intervention
   Questioning/strategy instruction 22 0.75* 0.58 to 0.92 8 0.48 0.18 to 0.77
  Text enhancements 6 0.62* 0.24 to 1.00 2 0.46 -0.2.98 to 3.90
   Fundamental reading skills 1 1.04 — 4 0.82 0.25 to 1.40
   instruction
   Other intervention 1 0.07 — 1 0.25 —
Treatment delivery
   Researchers 15 0.83* 0.62 to 1.04 4 0.52 -0.05 to 1.10
  Teachers 12 0.56 0.33 to 0.79 7 0.51 0.23 to 0.78
   Adult tutors 1 0.42 — 1 0.80 —
  Technology 2 0.66 -2.90 to 4.72 2 0.32 -3.11 to 3.76
Note: n of effect sizes = n of studies; ESw = weighted effect size; CI = confidence interval.
*Within-category (Qw) statistical significance, p < .05.

associated with very high effect sizes (> 2): Bakken et al. resulted in negative effect sizes. This study incorporated
(1997), Jitendra et al. (2000), Miranda, Villaescusa, and peer tutors using collaborative strategic reading (CSR). The
Vidal-Abarca (1997), Williams (1998), and Esser (2001). intervention was implemented by general education class-
All of these studies included teaching students to ask and room teachers, and the researchers indicated that a limitation
answer questions about the main idea and all but Williams of the study was that CSR was a more complex intervention
(1998) included some form of self-monitoring. In addition, than the comparison condition and took considerably longer
both Miranda et al. (1997) and Esser (2001) included attri- for teachers to teach students and integrate into reading
bution retraining components. Only one text enhancement practice. In addition, the sample in this study included stu-
study was associated with a very high effect size: Peverly dents whose fluency was far below grade level and therefore
and Wood (2001). may have benefited to a lesser degree from the complex read-
Three questioning/strategy instruction studies (Johnson, ing comprehension strategies taught. Finally, two questioning/
Graham, & Harris, 1997; Klingner & Vaughn, 1996; Vaughn, strategy instruction studies (Abdulaziz, 1999; Mastropieri,
Chard, Bryant, Coleman, & Kouzekanani, 2000) and one Scruggs, Hamilton, et al., 1996) and one “other” study
fundamental reading skills study (Rabren, Darch, & Eaves, (Laub, 1997) were associated with small effect sizes (< 0.2).
1999) were associated with negative effect sizes. Johnson Both of these questioning/strategy instruction studies
et al. (1997) had negative effect sizes both immediately fol- investigated elaborative interrogation for improving reading
lowing treatment and on a generalization measure; however, comprehension.
because of the comparison involved with this study, this out-
come may be misleading. The purpose of this investigation
was to investigate additional benefits of adding explicit self- Comparisons of Effect Sizes
regulation procedures (goal setting and/or self-instruction) for Instructional Variables
to story grammar strategy taught using a self-regulated strat- Delivery agent. Outcomes were evaluated for differences
egy development instructional model. Although all students among treatment delivery agents (e.g., researcher, adult
benefited from strategy instruction, differential benefits tutor, teacher) and are presented in Table 2. For criterion-
were not found for the additional self-regulation procedures referenced measures of treatment effects, the mean effect
investigated. Findings from the Klingner and Vaughn (1996) size for researchers (0.83) was higher than the mean effect
study also may be misleading because of the nature of the size for teachers (0.56). Although these differences fell
comparisons. The purpose of this study was to investigate slightly short of statistical significance, Q(1, N = 26) = 3.64,
outcomes of reciprocal teaching in different peer arrange- p = .056, cross-validation procedures revealed that the effect
ments (cross-age tutoring and cooperative learning). Like of delivery agent was quite robust across levels of other
the Johnson et al. study, students improved after the inter- study categories and explained much of the observed dif-
vention, but an advantage was not found between instruc- ferences in other comparisons, as described in the follow-
tional conditions with varying peer tutoring arrangements. ing sections. Substantial heterogeneity was found within
A study conducted by Vaughn, Chard, et al. (2000) also researcher-delivered treatments, however, indicating

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

430 Remedial and Special Education 31(6)

Table 3. Hedges’s d by Grade Level and Type of Measure

Criterion-Referenced Tests Norm-Referenced Tests

n ESw 95% CI n ESw 95% CI

Treatment measures
   Elementary 9 0.52* 0.26 to 0.77 7 0.55 0.25 to 0.86
   Middle or high 21 0.80 0.62 to 0.98 8 0.48 0.20 to 0.77
   Maintenance measures
   Elementary 4 0.26* -0.23 to 0.76 1 0.22 —
   Middle or high 3 1.53* 0.58 to 2.47 — — —
Generalization measures
   Elementary 2 -0.34 -3.52 to 2.84 — — —
   Middle or high 6 1.07 0.72 to 1.42 — — —
Note: n of effect sizes = n of studies; ESw = weighted effect size; CI = confidence interval.
*Within-category (Qw) statistical significance, p < .05.

considerable variability within this category. Statistical dif- associated with lower mean effect sizes than other mea-
ferences by treatment delivery agents were not observed for sures on norm-referenced tests, Q(3, N = 15) = 9.08, p = .03.
norm-referenced measures of treatment effects (p = .92). Within-category heterogeneity was not significant for any
Grade level. As seen in Table 3, mean treatment effects individual group. However, these effects were inconsistent
for criterion-referenced measures for middle and high across delivery agents and contained limited numbers of
school students (0.80) were larger than mean effects for observations.
elementary school students (0.52). These means were sta- Duration of treatment. Mean weighted effect sizes for
tistically different according to a homogeneity test, Q(1, criterion-referenced measures and norm-referenced tests by
N = 30) = 4.07, p = .04. Similar significant differences were treatment duration are reported in Table 4. For criterion-
also observed for maintenance, Q(1, N = 7) = 22.03, p < .01, referenced measures, mean weighted treatment effect sizes
and generalization measures, Q(1, N = 8) = 24.59, p < .01, were highest for treatments of medium duration (more than
although in these latter two cases the total number of stud- 1 week but less than 1 month). Differences among treat-
ies was limited. Significant heterogeneity was found within ments of varying length were statistically different accord-
many of these categories. In any case, cross-validation pro- ing to a homogeneity test, Q(2, N = 30) = 6.68, p = .04.
cedures revealed that the apparent difference in mean effect However, differences on norm-referenced tests by study
size by grade level was not a reliable effect and could be duration were not statistically significant (p = .83). That
explained by the influence of researcher-delivered treat- treatments of moderate length were associated with higher
ments in different grade levels. Grade-level effects were not effect sizes than either shorter or longer treatments is not
observed when collapsed across different levels of delivery easily explained; however, cross-validation procedures
agents. Neither were grade-level differences significant for indicated that this finding was robust across levels of other
norm-referenced tests (p = .68). variables, including delivery agent and setting. Because
Setting. Outcomes were evaluated for differences among researchers were not intervention agents in any longer-term
different types of setting (one to one, small group, class- treatments, however, it is possible that this variable played
rooms) and are presented in Table 4. Differences among some role in the observed outcomes.
weighted means by setting were statistically different for Other variables. Correlations were calculated between
criterion-referenced treatment measures, Q(2, N = 30) = 10.53, criterion-referenced treatment effect sizes and other study
p < .01 (although significant heterogeneity was observed variables. It was found that weighted effect size was not
within categories) but not for norm-referenced tests (p = .93). significantly correlated with the percentage of male partici-
Differences among settings types, when observed, could be pants in the sample or the mean reported sample IQ (p val-
explained by the differential presence of positive researcher- ues > .34). Comparisons were also made between studies that
delivered treatments in the small group category; teacher- included only students with LD and a smaller number of
delivered treatments did not vary appreciably by setting. studies that also included a minority of students who were
Type of passage. Weighted mean effect sizes by type of not characterized as having LD. The average mean treatment
passage are presented in Table 4. Weighted means associ- effect size for criterion-referenced measures for studies
ated with passage types approached but did not attain statis- including only students with LD (Mes = 0.63, n = 24) was
tical significance for criterion-referenced measures, Q(3, smaller than the average effect size for studies that also
N = 30) = 7.25, p = .07; however, expository text was included a minority of students (in any case, no more than one

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Berkeley et al. 431

Table 4. Hedges’s d (Treatment CRT) by Setting, Passage Type, Duration, and Design

Criterion-Referenced Tests Norm-Referenced Tests

n ESw 95% CI n ESw 95% CI

Setting
   One to one 6 0.50* 0.09 to 0.91 4 0.53 –0.15 to 1.21
   Small group 12 1.00* 0.75 to 1.26 5 0.58 0.14 to 1.03
   Classroom 12 0.55 0.34 to 0.77 6 0.48 0.17 to 0.78
Passage type
   Narrative 7 0.62* 0.30 to 0.94 4 0.88 0.36 to 1.40
   Expository 15 0.71* 0.50 to 0.92 7 0.32 –0.02 to 0.65
   Both 6 1.00 0.57 to 1.43 2 0.72 –2.45 to 3.50
   Unknown 2 0.26 –2.61 to 3.14 2 0.28 –2.27 to 2.82
Treatment duration
   Short (< 1 week) 7 0.48* 0.13 to 0.82 — — —
   Medium (> 1 week and 20 0.84* 0.66 to 1.03 7 0.49 0.14 to 0.84
   < 1 month)
   Long (> 1 month) 3 0.50 –0.17 to 1.17 8 0.53 0.27 to 0.79
Study design
   Experimental 20 0.90* 0.71 to 1.09 5 0.38 –0.10 to 0.86
   Quasi-experimental 9 0.46 0.22 to 0.70 8 0.45 0.18 to 0.72
   Pre-post 1 0.27 — 2 0.98 –1.74 to 3.69
Note: n of effect sizes = n of studies; ESw = weighted effect size; CI = confidence interval; CRT = criterion-referenced test
*Within-category (Qw) statistical significance, p < .05.

third) without LD (Mes = 0.88, n = 6), although this difference Fidelity of implementation. Studies were coded for reported
was not significant according to a homogeneity test, Q(1, N = treatment fidelity measures. For criterion-referenced treat-
30) = 2.56, p = .11. Examination of these outcomes by deliv- ment measures and norm-referenced treatment measures,
ery agent revealed that descriptive differences were associ- comparisons between weighted means for studies reporting
ated with differential effectiveness of researcher-implemented and not reporting fidelity of treatment measures were not sta-
treatments in the studies that included mixed samples. tistically significant (p values = .70 and .87, respectively).

Comparisons of Effect Sizes Use of Classroom Peers

Related to Study Quality Many studies that investigated questioning/strategy instruction
Study design. Table 4 displays mean weighted effect sizes also included peer mediation or self-regulation components.
by study design. For criterion-referenced tests, studies with Peer tutoring is designed to increase academic engagement
experimental (i.e., randomized) designs were associated and ownership of learning by students (Mastropieri, Scruggs,
with higher effect sizes than studies with quasi-experimental & Berkeley, 2007). Several of these peer tutoring interven-
designs. Differences among study designs were found to be tions incorporated components of questioning/strategy instruc-
statistically different according to a homogeneity test, Q(1, tion (e.g., activating prior knowledge, making predictions,
N = 29) = 9.94, p < .01. This observed outcome appeared to summarizing, identifying main ideas, clarifying, question-
be the consequence of disproportional numbers of researcher- ing, and analyzing text structure). These studies were also
delivered experimental treatments, with higher than average classified as questioning/strategy instruction. Examples
effect sizes in these investigations. No significant difference include (a) peer-assisted learning strategies (Calhoon, 2005;
on norm-referenced tests was observed between experimen- D. Fuchs, Fuchs, Mathes, & Simmons, 1997; L. S. Fuchs,
tal designs and quasi-experimental designs (p = .75), which Fuchs, & Kazdan, 1999; Saenz et al., 2005), (b) CSR
were associated with smaller numbers of observations and (Bryant et al., 2000; Vaughn, Chard, et al., 2000), (c) the
larger confidence intervals. The small number of pre-post Text Content and Structure Program (Lovett et al., 1996),
designs (n = 3) were associated with a descriptively lower (d) reciprocal teaching (e.g., Klingner & Vaughn, 1996),
than average effect size for experimental measures and and (e) classwide peer tutoring (Mastropieri et al., 2001).
higher than average effect sizes for norm-referenced mea- Effects for use of classroom peers are presented in Table 5
sures. However, none of these studies were extreme cases for criterion-referenced and norm-referenced treatment
(ES = 0.27, ES = 0.98, ES = 0.97). measures. The weighted mean treatment effect size from

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

432 Remedial and Special Education 31(6)

Table 5. Hedges’s d (Treatment Effects) by Peer Mediation and Self-Regulation Components

Criterion-Referenced Tests Norm-Referenced Tests

n ESw 95% CI n ESw 95% CI

Peer mediation component

  Yes 10 0.65 0.39 to 0.91 6 0.45 0.12 to 0.78
   No 20 0.72* 0.54 to 0.89 9 0.58 0.30 to 0.85
Self-regulation component
  Yes 8 0.92* 0.59 to 1.26 3 0.54 -1.29 to 2.37
   No 22 0.62* 0.46 to 0.78 10 0.34 -0.08 to 0.76
Note: n of effect sizes = n of studies; ESw = weighted effect size; CI = confidence interval.
*Within-category (Qw) statistical significance, p < .05.

criterion-referenced measures in studies employing class- instruction in self-instruction and goal setting was negative
room peers was only slightly smaller than the mean weighted compared to students receiving strategy instruction alone.
effect size from studies not employing classroom peers. The effect for goal setting and self-instruction was positive
These differences were not statistically significant accord- for a generalization measure in this same investigation.
ing to a homogeneity test, Q(1, N = 30) = 0.20, p = .65. However, overall only two studies assessed generalization
Significant heterogeneity was observed among outcomes of effects for treatments involving self-regulation, and the
treatments that did not include peer mediation. On norm- difference between these and generalization effects for
referenced measures, the weighted mean treatment effect treatments that did not involve self-regulation was not sig-
size for studies employing peers was also descriptively nificant (p = .79).
smaller than that for studies not employing peers, although
these differences also were not statistically significant,
Q(1, N = 14) = 0.53, p = .47. Discussion
The overall findings of this investigation are that interven-
tions on reading comprehension were generally very effec-
Self-Regulation tive, for both criterion-referenced measures (Mes = 0.70)
Several studies also included some type of self-regulation and norm-referenced tests (Mes = 0.52). The overall mean
component. Examples include (a) self-regulated strategy effect size, for both types of measures across treatment,
development combined with story grammar marker for nar- maintenance, and generalization outcomes, was 0.65, a fig-
rative text (Johnson et al., 1997), (b) attribution retraining ure most similar to that (0.72) reported by Swanson (1999)
combined with a combination of questioning strategies for reading comprehension intervention studies reported
(Manset-Williamson & Nelson, 2005; Miranda et al., 1997), between 1972 and 1997. Overall effect sizes reported by
(c) self-regulation strategies combined with reading com- Talbott et al. (1994), and Mastropieri, Scruggs, Bakken,
prehension strategies (Manset-Williamson & Nelson, 2005), et al. (1996) were larger (1.13 and 0.98, respectively) but
(d) self-regulation and the “RAP” strategy (Katims & Harris, nonetheless also revealed the overall effectiveness of read-
1997), and (e) self-monitoring combined with a main idea ing comprehension interventions. Although the present
strategy (Jitendra et al., 2000). results are based on different investigations reported since
Table 5 displays mean weighted effect sizes, with associ- the previous meta-analyses, one reason the present out-
ated frequencies and confidence intervals, for studies incor- comes most closely resemble those reported by Swanson
porating and not incorporating a self-regulation feature. As may be that some similar procedures were employed, for
can be seen, studies incorporating a self-regulation feature example, in employing only one effect size per study for
were associated with higher weighted mean effect sizes than analysis (although we computed these separately for differ-
were those that did not incorporate self-regulation. These ent outcomes) and using weighted effect sizes, which in
differences approached, but did not reach, statistical signifi- these investigations had the effect of generally lowering
cance according to a homogeneity test, Q(1, N = 30) = 3.52, effect size values. Furthermore, the Talbott et al. meta-
p = .06. The mean for the self-regulation effects was lowered analysis included effect sizes as large as 15.1 standard devi-
somewhat by the treatment outcome of a study by Johnson ation units, which may have inflated the overall obtained
et al. (1997), which did not employ a true control condition mean to some extent.
(all conditions received strategy instruction) and in which The general conclusion that can be drawn from the present
the gain for students receiving strategy instruction plus investigation is that an additional decade of research has

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Berkeley et al. 433

demonstrated the continuing effectiveness of a wide variety of also observed within researcher-delivered interventions, the
reading comprehension interventions, including fundamental overall effect was apparently sufficiently robust to have
reading instruction, text enhancements, and questioning/ influenced these other variables. In general, it could be
strategy instruction. In response to a possible concern that stated that statistical significance testing was less useful
these generally positive findings represent a kind of “dodo than cross-validation procedures in identifying important
bird verdict,” where “everybody has won and all must variables in this investigation. Further complicating statisti-
have a prize” (Carroll, 1865/1995, p. 33; see Parloff, 1984; cal significance testing was the fact that comparisons were
Rosenzweig, 1936; Swanson & Sachse-Lee, 2000), it should underpowered as a consequence of small sample sizes.
be considered that meta-analytic techniques have also under- In addition to the similarities, differences were observed
scored the ineffectiveness of many other types of special in the type of investigations that were reported over the past
education interventions, including perceptual training (Mes = decade. In the present set of research reports, we observed
0.08), diet interventions (Mes = 0.12), and modality training proportionally more norm-referenced measures; more whole
(Mes = 0.14; Forness, 2001). class and general education classroom (and fewer individu-
The present investigation also supported previous find- ally administered) interventions; more teacher-implemented,
ings that structured cognitive strategies were very effective; rather than researcher-implemented, treatments; fewer
however, in the present investigation these effects were not behavioral treatments; and more peer-mediated interven-
greater than other types of intervention, such as text enhance- tions than those identified in previous meta-analyses. Thus,
ments (cf. Mastropieri, Scruggs, Bakken, et al., 1996). it appears that more recent research on reading comprehen-
Furthermore, a variety of additional study characteristics, sion has moved into larger, whole class settings, imple-
such as peer mediation and self-regulation, although useful mented more often by teachers and classroom peers, and has
in individual studies, did not exert a differential effect on nevertheless documented overall effective outcomes.
overall study outcomes.
Large confidence intervals reveal variability within the
data set. In many cases this had to do with the effectiveness Implications for Research to Practice
of the control condition, not necessarily the fact that the inter- The results of the present meta-analysis, taken together
vention was not effective. Variability is commonly observed with the results of previous synthesis efforts on the same
in meta-analysis, and it may be because of a number of factors. topic, now including approximately 100 independent inves-
For example, another previously reported finding supported tigations, suggest that a variety of interventions are very
by the present investigation is that interventions implemented effective in improving reading comprehension of students
by researchers resulted in reliably higher effect sizes than with LD; on criterion-referenced and norm-referenced mea-
interventions implemented by teachers or other school per- sures; including cognitive strategies, text enhancements,
sonnel. This finding, reported in previous meta-analyses and behavioral treatments; in a variety of classroom set-
(Gajria et al., 2007; Mastropieri, Scruggs, Bakken, et al., tings; and across different treatment durations. These sub-
1996; Talbott et al., 1994; also see Mastropieri, Scruggs, stantial effect sizes suggest that, compared to traditional
Berkeley, & Graetz, in press), was seen to be robust across instruction, reading comprehension can be greatly improved
a variety of potentially compromising variables, such as set- by the application of these treatments.
ting, design, and grade level. Although the precise reasons for Although the types of treatments in these investigations
this outcome are not known for certain, one possibility is that varied considerably, most had in common an effort to teach
researchers implemented treatments with more precision and students to attend more carefully or to think more systemati-
intensity, not necessarily because they were better teachers cally about text as it was being read. This was true to some
but rather because they were more familiar with the particular extent whether the treatment involved basic reading skills
strategy in question and how it should best be implemented. training; whether it involved highlighting, outlining, illus-
If this explanation is true, it suggests that, with additional trating, and spatial or semantic feature organization of text;
training and perhaps more “ownership” of the instruction or whether students were asked to summarize text, predict
by teachers, these strategies could be even more effective. outcomes, provide main ideas, analyze text structure, or
The influence of this “intervention agent” variable also provide explanations for provided information. Particularly
explained several of the other statistically “significant” given the commonly observed failures of students with
findings. That is, the observed differences among different LD to spontaneously employ such strategies (e.g., Scruggs,
levels of variables such as grade level, passage type, dura- Bennion, & Lifson, 1985), the greatest overall implication
tion of treatment, study design, and inclusion of small num- is that systematically employing virtually any or all of these
bers of students without LD could be explained at least to techniques is very likely to improve students’ ability to con-
some extent by the presence of differential numbers of struct meaning from text.
researcher-implemented treatments within different levels Unfortunately, observational studies suggest that very
of other variables. Although significant heterogeneity was little specialized strategy instruction is presently taking

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

434 Remedial and Special Education 31(6)

place, particularly in general education settings, even when *Bryant, D. P., Vaughn, S., Linan-Thompson, S., Ugel, N., Hamff, A.,
special education teachers are present. Scruggs, Mastropieri, & Hougen, M. (2000). Reading outcomes for students with and
and McDuffie (2007) recently summarized qualitative stud- without reading disabilities in general education middle-school
ies of coteaching practice and concluded that systematic, content area classes. Learning Disability Quarterly, 23, 238-252.
strategic instruction intended to improve learning of aca- *Burns, G. L., & Kondrick, P. A. (1998). Psychological behavior-
demic skills or content was observed only infrequently in ism’s reading therapy program: Parents as reading therapists
these inclusive settings. Empirically validated strategies, for their children’s reading disability. Journal of Learning Dis-
however effective in research studies, are of little use if they abilities, 31, 278-285.
are not systematically implemented in a variety of class- *Calhoon, M. B. (2005). Effects of a peer-mediated phonological
room settings. skill and reading comprehension program on reading skill
Future research could provide more evidence on the acquisition for middle school students with reading disabilities.
effects of longer term implementations of reading compre- Journal of Learning Disabilities, 38, 424-433.
hension instruction on norm-referenced measures of read- Carr, S. C., & Thompson, B. (1996). The effects of prior knowl-
ing and the effectiveness of combinations of these treatments. edge and schema activation strategies on the inferential read-
Researchers also could investigate the reasons behind the ing comprehension of children with and without learning
observed “researcher-implementation” effect by studying disabilities. Learning Disability Quarterly, 19, 48-61.
this variable directly or noting in teacher implementation Carroll, L. (1995). Alice’s adventures in wonderland. London:
studies any observed barriers to systematic implementation. Anness. (Original work published 1865)
Finally, because these interventions are apparently infre- Cohen, J. (1988). Statistical power analysis for the behavioral
quently implemented in inclusive environments, future sciences (2nd ed.). New York: Academic Press.
research could address why strategy instruction is not com- *Conklin, J. T. (2000). The effect of presentation media on student
monly implemented in general education classrooms and reading comprehension. Dissertation Abstracts International,
what steps could be taken to improve current practice. At 61(09A), 3512. (UMI No. 9988426)
present, however, it does appear that the knowledge exists Cook, T. D., & Leviton, L. C. (1980). Reviewing the literature:
for greatly improving reading comprehension skills of stu- A comparison of traditional methods with meta-analysis.
dents with LD. Journal of Personality, 4, 449-472.
Cooper, H. (1998). Synthesizing research: A guide for literature
Declaration of Conflicting Interests reviews (3rd ed.). Thousand Oaks, CA: Sage.
The authors declared no potential conflicts of interest with respect *DiCecco, V. M., & Gleason, M. M. (2002). Using graphic
to the authorship and/or publication of this article. organizers to attain relational knowledge from expository text.
Journal of Learning Disabilities, 35, 306-320.
Financial Disclosure/Funding Dowrick, P. W., Kim-Rupnow, W. S., & Power, T. J. (2006). Video
The authors received no financial support for the research and/or feedforward for reading. Journal of Special Education, 39,
authorship of this article. 194-207.
Durkin, D. (1978-1979). What classroom observations reveal
References about reading comprehension instruction. Reading Research
References marked with an asterisk indicate studies included in Quarterly, 15, 481-553.
the meta-analysis. *Esser, M. M. S. (2001). The effects of metacognitive strategy
*Abdulaziz, T. M. (1999). The role of elaborative interrogation training and attribution retraining on reading comprehension in
in acquiring knowledge from expository prose passages for African-American students with learning disabilities. Disserta-
students with learning and behavior disorders. Dissertation tion Abstracts International, 67(7-A), 2340. (UMI No. 3021672)
Abstracts International, 61(01A), 132. (UMI No. 9957697) *Fagella-Luby, M. (2006). Embedded learning strategy instruc-
*Bakken, J. P., Mastropieri, M. A., & Scruggs, T. E. (1997). Read- tion: Story-structure pedagogy in secondary classes for diverse
ing comprehension of expository science materials and students learners. Dissertation Abstracts International, 67(2-A), 516.
with learning disabilities: A comparison of strategies. Journal (UMI No. 3207952)
of Special Education, 31, 300-325. Forness, S. R. (2001). Special education and related services: What
*Boyle, J. R. (1996). The effects of cognitive mapping strategy of have we learned from meta-analysis? Exceptionality, 9, 185-197.
the literal and inferential comprehension of students with mild *Fuchs, D., Fuchs, L. S., Mathes, P. G., & Simmons, D. C. (1997).
disabilities. Learning Disability Quarterly, 12, 86-99. Peer-assisted learning strategies: Making classrooms more
*Boyle, J. R. (2000). The effects of a Venn diagram strategy on the responsive to diversity. American Educational Research Journal,
literal, inferential, and relational comprehension of students 34, 174-206.
with mild disabilities. Learning Disabilities: A Multidisciplinary *Fuchs, L. S., Fuchs, D., & Kazdan, S. (1999). Effects of peer-
Journal, 10, 5-13. assisted learning strategies on high school students with

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

Berkeley et al. 435

serious reading problems. Remedial and Special Education, *Laub, C. M. (1997). Effectiveness of Project Read on word attack
20, 309-318. skills and comprehension for third and fourth grade students
Gajria, M., Jitendra, A. K., Sood, S., & Sacks, G. (2007). Improving with learning disabilities. Dissertation Abstracts International,
comprehension of expository text in students with LD: A research 36(01), 0036. (UMI No. 1386289)
synthesis. Journal of Learning Disabilities, 40, 210-225. *Lovett, M. W., Borden, S. L., Warren-Chaplin, P. M., Lacerenza, L.,
Gardill, M. C., & Jitendra, A. K. (1999). Advanced story map DeLuca, L., & Giovinazzo, R. (1996). Text comprehension train-
instruction: Effects on the reading comprehension of students ing for disabled readers: An evaluation of reciprocal teaching and
with learning disabilities. Journal of Special Education, 33, text analysis training programs. Brain and Language, 54, 447-480.
2-17, 28. *MacArthur, C. A., & Haynes, J. B. (1995). Student Assistant for
Gersten, R., Fuchs, L. S., Williams, J. P., & Baker, S. (2001). Teach- Learning from Text (SALT): A hypermedia reading aid. Journal
ing reading comprehension strategies to students with learn- of Learning Disabilities, 28, 150-159.
ing disabilities: A review of research. Review of Educational *Manset-Williamson, G., & Nelson, J. M. (2005). Balanced,
Research, 71, 279-321. strategic reading instruction for upper-elementary and middle
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in school students with reading disabilities: A comparative study
social research. Beverly Hills, CA: Sage. of two approaches. Learning Disability Quarterly, 28, 59-74.
Hedges, L., & Olkin, I. (1985). Statistical methods for meta-analysis. Mastropieri, M. A., & Scruggs, T. E. (1997). Best practices in pro-
New York: Academic. moting reading comprehension in students with learning disabili-
Heise, K. (2004). The effects of the Read Naturally program on flu- ties. Remedial and Special Education, 18, 197-213.
ency, accuracy, comprehension, and student motivation in stu- Mastropieri, M. A., Scruggs, T. E., Bakken, J. P., & Whedon, C.
dents with learning disabilities. Masters Abstracts International, (1996). Reading comprehension: A synthesis of research in
42(06), 1957. (UMI No. 1419865) learning disabilities. In T. E. Scruggs & M. A. Mastropieri (Eds.),
Jenkins, J. R., Fuchs, L. S., van de Broek, P., Espin, C., & Deno, S. L. Advances in learning and behavioral disabilities: Intervention
(2003). Accuracy and fluency in list and context reading of skilled research (Vol. 10, pt. B, pp. 201-227). Oxford, UK: Elsevier.
and RD groups: Absolute and relative performance levels. Learn- Mastropieri, M. A., Scruggs, T. E., & Berkeley, S. (2007). Peers
ing Disabilities Research and Practice, 18, 237-245. helping peers. Educational Leadership, 64(5), 54-58.
*Jitendra, A. K., Hoppes, M. K., & Xin, Y. P. (2000). Enhancing Mastropieri, M. A., Scruggs, T. E., Berkeley, S., & Graetz, J. (in
main idea comprehension for students with learning problems: press). Does special education improve learning of secondary
The role of a summarization strategy and self-monitoring content? A meta-analysis. Remedial and Special Education.
instruction. Journal of Special Education, 34, 127-139. Mastropieri, M. A., Scruggs, T. E., & Graetz, J. E. (2003). Reading
*Johnson, L., Graham, S., & Harris, K. R. (1997). The effects of goal comprehension instruction for secondary students: Challenges for
setting and self-instructions on learning a reading comprehension struggling students and teachers. Learning Disability Quarterly,
strategy: A study with students with learning disabilities. Journal 26, 103-116.
of Learning Disabilities, 30, 80-91. *Mastropieri, M. A., Scruggs, T. E., Hamilton, S. L., Wolfe, S.,
*Katims, D. S., & Harris, S. (1997). Improving the reading com- Whedon, C., & Canevaro, A. (1996). Promoting thinking skills
prehension of middle school students in inclusive classrooms. of students with learning disabilities: Effects on recall and com-
Journal of Adolescent and Adult Literacy, 41, 116-123. prehension of expository prose. Exceptionality, 6, 1-11.
*Kim, A. (2002). Effects of computer-assisted collaborative strate- *Mastropieri, M. A., Scruggs, T. E., Mohler, L., Beranek, M.,
gic reading on reading comprehension for high-school students Boon, R., Spencer, V., et al. (2001). Can middle school students
with learning disabilities. Dissertation Abstracts International, with serious reading difficulties help each other and learn any-
64(11A), 4009. (UMI No. 3110632) thing? Learning Disabilities Research and Practice, 16, 18-27.
*Kim, A., Vaughn, S., Klingner, J. K., Woodruff, A. L., *Miranda, A., Villaescusa, M. I., & Vidal-Abarca, E. (1997). Is
Reutebuch, C. K., & Kouzekanani, K. (2006). Improving the attribution retraining necessary? Use of self-regulation proce-
reading comprehension of middle school students with disabil- dures for enhancing the reading comprehension strategies of
ities through computer-assisted collaborative strategic reading. children with learning disabilities. Journal of Learning Dis-
Remedial and Special Education, 27, 235-249. abilities, 30, 503-512.
Kim, A., Vaughn, S., Wanzek, J., & Wei, S. (2004). Graphic *Mothus, T. G. (1997). The effects of strategy instruction on
organizers and their effects on the reading comprehension of the reading comprehension achievement of junior secondary
students with LD: A syntheses of research. Journal of Learning school students. Dissertation Abstracts International, 42(01),
Disabilities, 37, 105-118. 44. (UMI No. 766434601)
*Klingner, J. K., & Vaughn, S. (1996). Reciprocal teaching of *Newbern, S. L. (1998). The effects of instructional settings on the
reading comprehension strategies for students with learning efficacy of strategy instruction for students with learning dis-
disabilities who use English as a second language. Elementary abilities. Dissertation Abstracts International, 59(05A), 1527.
School Journal, 96, 275-293. (UMI No. 9832950)

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014

436 Remedial and Special Education 31(6)

*Oakland, T., Black, J. L., Stanford, G., Nussbaum, N. L., & Swanson, H. L., Hoskyn, M., & Lee, C. (1999). Interventions for
Balise, R. R. (1998). An evaluation of the dyslexia training students with learning disabilities: A meta-analysis of treatment
program: A multisensory method for promoting reading in outcomes. New York: Guilford.
students with reading disabilities. Journal of Learning Dis- Swanson, H. L., & Sachse-Lee, C. (2000). A meta-analysis of
abilities, 31, 140-147. single-subject research for students with LD. Journal of Learn-
Parloff, M. B. (1984). Psychotherapy research and its incredible ing Disabilities, 33, 114-136.
credibility crisis. Clinical Psychology Review, 4, 95-109. Talbott, E., Lloyd, J. W., & Tankersley, M. (1994). Effects of read-
*Peverly, S. T., & Wood, R. (2001). The effects of adjunct ques- ing comprehension interventions for students with learning
tions and feedback on improving the reading comprehension disabilities. Learning Disability Quarterly, 17, 223-232.
skills of learning-disabled adolescents. Contemporary Educa- Thompson, B. (2006). Research synthesis: Effect sizes. In J. Green,
tional Psychology, 26, 25-43. G. Camilli, & P. B. Elmore (Eds.), Handbook of complementary
*Rabren, K., Darch, C., & Eaves, R. C. (1999). The differential methods in education research (pp. 583-603). Washington, DC:
effects of two systematic reading comprehension approaches American Educational Research Association.
with students with learning disabilities. Journal of Learning *Torgesen, J. K., Alexander, A. W., Wagner, R. K., Rashotte, C. A.,
Disabilities, 32, 36-47. Voeller, K. K. S., & Conway, T. (2001). Intensive reme-
*Rankhorn, B., England, G., Collins, S. M., Lockavitch, J. F., & dial instruction for children with severe reading disabilities:
Algozzine, B. (1998). Effects of the failure free reading pro- Immediate and long-term outcomes from two instructional
gram on students with severe reading disabilities. Journal of approaches. Journal of Learning Disabilities, 34, 33-58.
Learning Disabilities, 31, 307-312. *Ugel, N. S. (1999). The effects of a multicomponent reading inter-
*Rooney, J. (1997). The effects of story grammar strategy training vention on the reading achievement of middle school students
on the story comprehension, self-efficacy and attributions of with reading disabilities. Dissertation Abstracts International,
learning disabled students. Dissertation Abstracts International, 61(01A), 118. (UMI No. 9959597)
58(5-A), 1642. (UMI No. 9732965) *Vaughn, S., Chard, D. J., Bryant, D. P., Coleman, M., &
Rosenberg, M. S., Adams, D. C., & Gurevitch, J. (2000). Kouzekanani, K. (2000). Fluency and comprehension inter-
MetaWin: Statistical software for meta-analysis. Sunderland, ventions for third-grade students. Remedial and Special Educa-
MA: Sinauer Associates. tion, 21, 325-335.
Rosenzweig, S. (1936). Some implicit common factors in diverse Vaughn, S., Gersten, R., & Chard, J. D. (2000). The underlying
methods of psychotherapy. American Journal of Orthopsy- message in LD intervention research: Findings from research
chiatry, 6, 412-415. synthesis. Exceptional Children, 67, 99-114.
*Saenz, L. M., Fuchs, L. S., & Fuchs, D. (2005). Peer-assisted Vaughn, S., Levy, S., Coleman, M., & Bos, C. S. (2002). Reading
learning strategies for English language learners with learning instruction for students with LD and EBD: A synthesis of obser-
disabilities. Exceptional Children, 71, 231-247. vation studies. Journal of Special Education, 36, 2-13.
Scruggs, T. E., Bennion, K., & Lifson, S. (1985). Learning disa- *Williams, J. P. (1998). Improving the comprehension of disabled
bled students’ spontaneous use of test-taking skills on reading readers. Annals of Dyslexia, 48, 213-238.
achievement tests. Learning Disability Quarterly, 8, 205-210. *Xin, J. F., & Rieth, H. (2001). Video-assisted vocabulary instruc-
Scruggs, T. E., Mastropieri, M. A., & McDuffie, K. A. (2007). Co- tion for elementary school students with learning disabili-
teaching in inclusive classrooms: A meta-synthesis of qualita- ties. Information Technology in Childhood Education Annual,
tive research. Exceptional Children, 73, 392-416. 1, 87-103.
Sideridis, G. D., Mouzaki, A., Simos, P., & Protopapas, A. (2006).
Classification of students with reading comprehension dif-
ficulties: The roles of motivation, affect, and psychopathology. About the Authors
Learning Disability Quarterly, 29, 159-180. Sheri Berkeley, PhD, was an assistant professor in the Department
*Stevens, R. J., & Slavin, R. E. (1995). The cooperative elemen- of Communication Sciences and Special Education at the Univer-
tary school: Effect on student achievement and social relations. sity of Georgia in Athens, GA, at the time of writing. She is now an
American Educational Research Journal, 32, 321-351. assistant professor in the College of Education and Human Devel-
Swanson, H. L. (1999). Reading research for students with LD: opment, George Mason University in Fairfax, VA.
A meta-analysis of intervention outcomes. Journal of Learning
Disabilities, 32, 504-532. Thomas E. Scruggs, PhD, is university professor and director of
Swanson, H. L., & Hoskyn, M. (2000). Intervention research the PhD in education program, College of Education and Human
for students with learning disabilities: A comprehensive Development, George Mason University in Fairfax, VA.
meta-analysis of group design studies. In T. E. Scruggs &
M. A. Mastropieri (Eds.), Advances in learning and behavioral Margo A. Mastropieri, PhD, is university professor, College of
disabilities: Vol. 14. Educational interventions (pp. 1-153). Education and Human Development, George Mason University in
Oxford, UK: Elsevier. Fairfax, VA.

Downloaded from rse.sagepub.com at CARLETON UNIV on June 30, 2014