Journal of Experimental Psychology: Copyright 2005 by the American Psychological Association

Learning, Memory, and Cognition 0278-7393/05/$12.00 DOI: 10.1037/0278-7393.31.5.1155

2005, Vol. 31, No. 5, 1155–1159

The Positive and Negative Consequences of Multiple-Choice Testing

Henry L. Roediger III Elizabeth J. Marsh

Washington University in St. Louis Duke University

Multiple-choice tests are commonly used in educational settings but with unknown effects on students’
knowledge. The authors examined the consequences of taking a multiple-choice test on a later general
knowledge test in which students were warned not to guess. A large positive testing effect was obtained:
Prior testing of facts aided final cued-recall performance. However, prior testing also had negative
consequences. Prior reading of a greater number of multiple-choice lures decreased the positive testing
effect and increased production of multiple-choice lures as incorrect answers on the final test. Multiple-
choice testing may inadvertently lead to the creation of false knowledge.

Keywords: testing effect, negative suggestion effect, false memory, multiple-choice testing

Multiple-choice testing is pervasive in university education. different material than do students expecting an essay exam (Rick-
Many large introductory courses in natural and social sciences rely ards & Friedman, 1978). Work in other traditions within cognitive
on multiple-choice tests (and to a lesser extent, true–false testing) psychology suggests that there may be negative effects of testing.
as the primary means of student assessment. Furthermore, because For example, a multiple-choice test is structurally similar to the
good multiple-choice tests are so difficult to construct, the same misinformation paradigm begun by Loftus and Palmer (1974), a
questions are often used across semesters. The result is that the test naturalistic variation of the classic A–B, A–D interference para-
bank needs to be protected, meaning that many professors neither digm (e.g., McGeoch, 1932; see Crowder, 1976, chapter 8 for a
review the test in class nor return the tests to students. Rather, review). In these studies, subjects typically see a simulated acci-
professors let students review the tests in their offices, although in dent or crime scene and later receive either a test or a narrative
reality few students do so. Thus, in many classes, students take about the scene that contains a few items of misinformation.
multiple-choice tests and receive no feedback except for their Producing or choosing misinformation on an initial test has neg-
overall score; many students never review their wrong answers and ative consequences for later memory (see Ackil & Zaragoza, 1998;
correct their misconceptions. The experiment we report examines Meade & Roediger, in press; Roediger, Jacoby, & McDermott,
the effects that this practice may have on students’ knowledge as 1996; Roediger, Wheeler, & Rajaram, 1993; Schooler, Foster, &
displayed on later tests (such as a cumulative final exam) and the Loftus, 1988; Zaragoza, Payment, Ackil, Drivdahl, & Beck, 2001).
knowledge that students may carry away from the course. Like the eyewitness paradigm, both multiple-choice and true–false
There are many advantages to multiple-choice testing. Although tests routinely expose students to wrong answers (misinformation).
difficult to create, they are easy to score and therefore are the For example, in a standard four-alternative multiple-choice test,
evaluation method of choice in large classes. The added benefit is three alternatives are wrong, and only one is correct. If subjects
that taking a test generally improves students’ performance on a read all four statements carefully, then they are exposed to three
later test; this is referred to as the testing effect (Bjork, 1975; statements containing wrong answers (misinformation) and one
Carrier & Pashler, 1992; Glover, 1989; Hogan & Kintsch, 1971; containing the correct answer. Even if subjects pick the correct
Izawa, 1970; Kuo & Hirshman, 1996; McDaniel & Masson, 1985; answer at the time, reading the wrong statements may make them
Runquist, 1986; Spitzer, 1939; Thompson, Wenger, & Bartling, later seem true, by extension from other research. That is, simply
1978; Tulving, 1967). repeating statements has been shown in many studies to increase
However, there can be negative consequences of relying exclu- the probability that the statements will be judged true (e.g., Bacon,
sively on multiple-choice tests. Students expecting a multiple- 1979; Begg, Armour, & Kerr, 1985; Hasher, Goldstein, & Top-
choice test (relative to an essay test) spend less time studying for pino, 1977).
the test (Kulhavey, Dyer, & Silver, 1975), and they take notes on The idea that multiple-choice tests teach misinformation has not
gone unnoticed in experimental circles. For example, Toppino and
his colleagues have shown that when students take true–false and
Henry L. Roediger III, Department of Psychology, Washington Univer- multiple-choice tests, the lures are later judged as more true than
sity in St. Louis; Elizabeth J. Marsh, Psychological and Brain Sciences, novel false facts (Toppino & Brochin, 1989; Toppino & Luipers-
Duke University. beck, 1993; see also Rees, 1986), although the rated truth of
This research was supported by the Collaborative Activity Award from
repeated falsehoods never reaches the level of objectively true
the James S. McDonnell Foundation. We thank Lisa Fazio for her help in
collecting the data, Elaina Pelky for help in scoring the data, and Holli Sink
statements. Remmers and Remmers (1926) coined the phrase “the
for assistance with article preparation. negative suggestion effect” to refer to students’ increased belief in
Correspondence concerning this article should be addressed to Henry L. false statements from the tests. Similarly, Brown (1988) and Ja-
Roediger III, Department of Psychology, Washington University, Campus coby and Hollingshead (1990) showed that exposure to misspelled
Box 1125, St. Louis, MO 63130-4899. E-mail: roediger@artsci.wustl.edu words led subjects to misspell the words later on a traditional oral

1155
1156 OBSERVATIONS

spelling test, again showing a negative suggestion effect. Brown, Rainier) and animals (e.g., sea otters). Passages were only chosen if they
Schilling, and Hockensmith (1999) found that exposing subjects to allowed the creation of four items, each of which could be presented in all
misinformation after an initial test influenced subjects on the final four test formats necessary for the design (2, 4, and 6 alternative multiple-
cued-recall and multiple-choice tests. This outcome occurred even choice, plus cued recall). To create the multiple-choice questions, we
generated five plausible lures. Two lures were randomly removed to create
when wrong information was identified as such during the inter-
the four-alternative version, and two additional lures were randomly re-
polated task. However, performance was unaffected by the number
moved to create the two-alternative version. Across subjects, the four
of incorrect alternatives, the number of presentations, and the questions corresponding to each of the passages were rotated through the
length of the retention interval. four multiple-choice conditions (0 [not tested], 2, 4, or 6 alternatives).
We examined the possible effects of negative suggestion from The passages were divided into two groups of 18 passages; passages on
multiple-choice tests in situations resembling those of introductory similar subjects were placed in different groups. One half of the subjects
classes outlined in the first paragraph. Students read one half of a read the first set of passages, and one half of the subjects read the second
collection of educationally relevant materials (nonfiction passages set of passages; all subjects were tested on the complete set. This feature
about science and history). They then took a first multiple-choice realized the read–not read aspect of the design.
test (with no feedback) and later a second test, a cued-recall test The multiple-choice test contained 144 questions, 108 of which corre-
sponded to the critical questions. The final cued-recall test contained 216
with a warning against guessing. When subjects were tested on
questions, 144 of which corresponded to the critical questions, and the
passages they had not read, this corresponded to testing in classes
remainder were fillers. Both tests were given in paper-and-pencil format.
in which students have not done the relevant reading (and also lead
to a difficulty-of-question manipulation, because the items were
necessarily answered less well in the condition in which the Procedure
passages had not been read). The experiment had four phases: passage reading, the multiple-choice
On the basis of the literatures of the testing effect and interfer- test, the visuospatial-reasoning filler task, and the final cued-recall test.
ence effects, we predicted that overall a multiple-choice test (even In Phase I, passage reading, subjects read 18 of the 36 passages. The
without feedback) would have a positive effect on a later cued- amount of time devoted to each passage was determined in pretesting; on
recall test (the standard testing effect). However, we also expected average, subjects were given up to 90 s to read each passage. The goal was
that the number of lures on the multiple-choice test would nega- for all subjects to finish reading each passage once. Subjects were given a
tively affect performance: The more alternatives on the multiple- reading recording sheet on which they indicated when they had completed
reading the passage; the experimenter monitored the subjects for comple-
choice test, the worse performance on the later cued-recall test and
tion and moved them to the next passage when all had finished reading.
the smaller the positive testing effect. More important, we pre- In Phase II, subjects took the multiple-choice test. Prior to the test, they
dicted that an increased number of alternatives on the multiple- were instructed to provide a response to each and every question, even if
choice test would also increase errors on the later cued-recall test they had to guess. They were informed that some of the questions would
(similar to a fan effect; Lewis & Anderson, 1976). Furthermore, have two possible answers, some four, and others six. The experimenter
we examined whether the negative effect of more alternatives instructed them to read each question carefully, being sure to read all
would interact with the difficulty of the items, such that increasing possible answers, and then to circle the best answer. They were told to
the number of alternatives would cause a greater negative effect on answer the questions in the order in which they appeared, and not to go
later cued recall for difficult items. The reason is that in the back and change answers. Subjects were given up to 22 min to answer 144
literature on retroactive interference, greater interference from questions, and the experimenter advised them how time was passing.
Subjects who finished early were instructed to turn over their tests and wait
A–D learning occurs when A–B learning is less complete (Crow-
quietly for the next set of instructions. No feedback was given as to
der, 1976; Postman, 1962). correctness of answers.
Phase III involved the filler task, the visuospatial brainteasers. Subjects
Method were given 5 min to work on a series of puzzles.
Phase IV involved the cued-recall test. Subjects were strongly warned
Subjects not to guess just before they took the final cued-recall test. They were told
to answer each question only if they were reasonably sure of the answer,
A total of 24 undergraduates from Washington University participated in
and to draw a line through the answer space if they did not know the
the experiment for partial fulfillment of a course requirement. Subjects
answer (as opposed to guessing). Subjects were allotted up to 35 min to
were tested either individually or in groups of up to 5 people.
answer 216 questions, and the experimenter kept them aware of how time
was passing. As with the multiple-choice test, subjects who finished early
Design were instructed to turn over their tests and wait quietly for the next set of
instructions. At the end of the experiment, subjects were debriefed and
The experiment had a 2 (passage status: read or not) ⫻ 4 (number of thanked for their participation.
alternatives on the multiple-choice test: 0, 2, 4, 6) design. The dependent
measures were proportion correct answers and proportion errors on the
final cued-recall test. Results
All results were significant at the .05 level unless otherwise
Materials noted.
We used 36 nonfiction passages from the reading comprehension sec-
tions of the Test of English as a Foreign Language and Graduate Record Multiple-Choice Test
Examination practice test books. These passages spanned a variety of
topics, including famous people (e.g., Louis Armstrong), science (e.g., the The multiple-choice test data are shown in Table 1. As expected,
sun), history (e.g., the founding of New York City), places (e.g., Mount more questions were answered correctly when subjects had read
 OBSERVATIONS 1157

Table 1 cued-recall test. Even though subjects received a strong warning

Proportion Correct on the Multiple-Choice Test as a Function against guessing, they did report prior multiple-choice lures as
of Whether Subjects Read the Passages and Number of answers on the cued-recall test. The relevant data are shown in the
Alternatives (Including Correct) for Each Question bottom portion of Table 2. Production of wrong answers increased
linearly with number of previously read alternatives for both read,
No. of Alternatives F(1, 23) ⫽ 7.58, MSE ⫽ .01, and nonread passages, F(1, 23) ⫽
Passages Two Four Six
9.96, MSE ⫽ .01. Put another way, prior testing increased produc-
tion of multiple-choice lures on the cued-recall tests, F(3, 69) ⫽
Read .90 .77 .74 5.86, MSE ⫽ .01, and this effect did not interact with whether the
Not read .72 .50 .39 passage had been read. Because reading did not interact with prior
number of alternatives, we collapsed over the reading variable to
Note. The standard error was .015 in the read condition and .013 when
the passages were not read. compare performance in each of the prior multiple-choice condi-
tions (0, 2, 4, 6) with each other. Of the six pairwise comparisons,
four reached traditional levels of significance. The difference
the relevant passages, F(1, 23) ⫽ 362.68, MSE ⫽ .01. Also as
between zero (M ⫽ .05) and two (M ⫽ .07) was marginally
expected, subjects answered more questions correctly when tested
significant, t ⫽ 1.57, SE ⫽ .01, and the difference between four
with fewer alternatives, F(2, 46) ⫽ 102.51, MSE ⫽ .01. Reading
and six was not significant.
interacted significantly with number of alternatives, F(2, 46) ⫽
Finally, we examined whether the misleading lures produced on
6.18, MSE ⫽ .01. For read passages, subjects correctly answered
the final test were the same or different from the options that
more two-alternative than four-alternative questions, t(23) ⫽ 3.59,
subjects had selected on the first (multiple-choice) test. Of the
SE ⫽ .03, but they performed no worse when tested with six
errors produced on the final cued-recall test, 75% had been se-
alternatives than with four (t ⬍ 1). When subjects had not read the
lected on the initial test. The effect of number of prior multiple-
relevant passages, performance decreased even further when the
choice alternatives remained significant when the analysis was
number of alternatives was increased from four to six, t(23) ⫽
restricted to errors that had also been made on the earlier multiple-
3.48, SE ⫽ .03. In summary, subjects did worse on the multiple-
choice test, F(2, 46) ⫽ 4.94, MSE ⫽ .01. In contrast, the linear
choice test when tested with more alternatives and with unfamiliar
trend disappeared when the analysis was restricted to intrusions of
(more difficult) material.
lures that had not been chosen on the earlier test (Fs ⬍ 1). Thus,
production of the lures on the final test was largely due to subjects
Cued-Recall Test: Correct Answers reproducing the incorrect lures that they had chosen on the earlier
Of greatest interest was whether the prior multiple-choice for- multiple-choice test. Multiple-choice lures that were read but not
mat affected cued-recall performance even when subjects were selected were not produced on the final test, although perhaps the
strictly warned against guessing on the final test. Despite the familiarity accrued to these items would be expressed on other
warning, as shown in the top portion of Table 2, the number of forms of testing, such as true–false.
prior multiple-choice alternatives had two different, opposite ef-
fects on later recall. Cued-Recall Test: Production of Other Errors and Items
There was a large positive testing effect, F(1, 23) ⫽ 61.52, Left Unanswered
MSE ⫽ .01, and this was larger for read passages, F(1, 23) ⫽ 8.79,
Given that response categories are not independent, it is not
MSE ⫽ .01. On average, subjects answered 28% of questions
surprising that fewer questions were left unanswered when the
correctly for nontested items but were able to answer 46% cor-
rectly when the questions had been tested previously.
Not all forms of prior testing were equally beneficial. Prior Table 2
testing with two alternatives led to 51% correct on the cued-recall Performance on the Cued Recall Test as a Function of Passage
test; this dropped to 45% following testing with four alternatives, Reading Status and Number of Alternatives (Including the
and 43% following testing with six alternatives. The effect of Correct Answer) on the Prior Multiple-Choice Test
number of prior alternatives remained significant even after the
never-tested items were removed from the analysis, F(2, 46) ⫽ No. of previous multiple-choice
alternatives
6.35, MSE ⫽ .01, and there was no interaction between reading
and number of prior alternatives (Fs ⬍ 1). That is, performance Zero
decreased linearly with number of alternatives previously read for Passage reading status (not tested) Two Four Six
both passages that had been read, F(1, 23) ⫽ 4.05, MSE ⫽ .01, p ⫽
Proportion correct
.056, and for those that had not been read, F(1, 23) ⫽ 6.25, MSE ⫽ Read passages .40 .67 .61 .61
.01. All pairwise comparisons were significant except the differ- Not read passages .16 .34 .28 .26
ence between having been tested with four (M ⫽ .45) versus six Proportion lure errors
(M ⫽ .43) alternatives. Read passages .04 .06 .08 .09
Not read passages .06 .09 .13 .15

Cued-Recall Test: Production of Multiple-Choice Lures Note. For correct responses, the standard error was .022 in the read
condition and .015 in the nonread condition. For multiple-choice lure
The critical prediction concerned whether subjects would report responses, the standard error was .009 in the read condition and .018 in the
lures from the multiple-choice test as correct answers on the nonread condition.
1158 OBSERVATIONS

relevant passages had been read, F(1, 23) ⫽ 168.75, MSE ⫽ .03. items may simply have increased the familiarity of the correct
Similarly, prior testing reduced unanswered items on the final test, answer, especially if subjects guessed it was the correct answer.
F(3, 69) ⫽ 25.16, MSE ⫽ .02. Both reading and prior testing The most important finding from our study is that the lures on
increased correct answers and, thus, reduced items left multiple-choice tests can be a source of interference in learning
unanswered. from tests. The greater the number of alternatives provided on the
Subjects did intrude wrong answers other than the multiple- multiple-choice test, the less benefit subjects received from taking
choice lures, albeit at a relatively low rate (M ⫽ .05), which was the test. Besides reducing correct responding, increasing the num-
expected given the warning against guessing. These extraneous ber of alternatives also increased the probability that subjects
errors were more frequent when subjects had not read the passages answered cued-recall questions with lures from the prior multiple-
(M ⫽ .06) than when they had (M ⫽ .04), F(1, 23) ⫽ 13.63, choice test. Thus, multiple-choice tests with many lures can actu-
MSE ⫽ .01, and they also decreased as a function of the number ally create false knowledge or beliefs in students—false beliefs
of alternatives on the initial test, F(3, 69) ⫽ 11.29, MSE ⫽ .01. that may be carried from the classroom. We are currently inves-
Errors decreased after testing with two (M ⫽ .04), four (M ⫽ .04), tigating the effects of delay and whether negative effects of testing
or six (M ⫽ .03) alternatives, as compared with the nontested persist if students receive feedback on the multiple-choice
condition (M ⫽ .09). The drop in extraneous errors in the tested responses.
conditions was accompanied by increases in production of both How does a multiple-choice test impair performance on a later
correct and multiple-choice lure answers, as described earlier. test? Not only does reading the multiple-choice question serve as
The last analysis considered errors as a function of opportunities a study trial for the correct answer but it also exposes the subject
to produce errors. That is, as learning increased in the tested to one or more incorrect answers, similar to retroactive interfer-
conditions, opportunities to make an error were reduced. A 2 ence (McGeoch, 1932), a misinformation effect (Loftus & Palmer,
(passage status: read or not) ⫻ 2 (number of alternatives: 0, 2, 4, 1974), or a fan effect (Lewis & Anderson, 1976). As such, the test
6) analysis of variance on total errors conditional on the failure to may remind subjects of prior wrong beliefs that they otherwise
answer correctly revealed a strong tendency for prior testing to would not retrieve, or it may also teach subjects incorrect answers
increase errors, F(3, 69) ⫽ 3.82, MSE ⫽ .03. When subjects did as they incorrectly use knowledge to eliminate the other answers
not give the correct answer, they produced a wrong answer on 20% (including the correct one). Reading the lures also increases the
of trials in the nontested condition, as compared with 26% of items fluency or familiarity of the incorrect answers and may result in
previously tested with two alternatives, and 30% of items previ- the “mere truth” effect from statement repetition (e.g., Hasher et
ously tested with four or six alternatives. al., 1977). In short, the same mechanisms that may help a subject
to correctly produce an answer later on may also lead to production
of the incorrect target lures.
Discussion In sum, the effect of taking a multiple-choice test has two
opposing effects on students’ knowledge. Despite an overall pos-
Taking a multiple-choice test caused subjects to answer later itive testing effect, students will sometimes come to believe that
cued-recall tests with incorrect information. The effect was greater the distracter answers are correct and therefore leave the exam
for the nonstudied material and also increased with the number of having acquired false knowledge. Our experiment and the few
lures on the multiple-choice test. This negative suggestion effect prior ones similar to it (e.g., Brown et al., 1999; Toppino &
occurred despite the fact that subjects were strongly warned Luipersbeck, 1993) are just the beginnings of research that will
against guessing. However, there was also a positive effect of determine effects of testing under conditions that are somewhat
testing: Subjects answered more questions correctly on the final realistic for educational concerns. We mimicked the situation in
cued-recall test when they had been tested previously on a which students study (or perhaps fail to study) class material and
multiple-choice test relative to when they had not. This positive then take quizzes before a test, or take a series of tests before a
effect of testing occurred even though no feedback was given final exam. In our opinion, the positive and negative effects of
about student’s selections on the multiple-choice test. This out- multiple-choice tests uncovered in our experiment deserve wide
come confirms that the testing effect frequently observed in list discussion at all levels of education in which multiple-choice tests
learning experiments (e.g., Tulving, 1967) applies to educationally are widely used.
relevant materials (see Roediger & Karpicke, in press, for further
evidence with prose material and free-recall tests). Relatively few
References
prior studies have used such materials (however, see Glover, 1989;
McDaniel & Fisher, 1991; Spitzer, 1939), and none used a design Ackil, J. K., & Zaragoza, M. S. (1998). Memorial consequences of forced
involving transfer from multiple-choice to cued-recall tests, as we confabulation: Age differences in susceptibility to false memories. De-
did. velopmental Psychology, 34, 1358 –1372.
Prior reading of multiple-choice alternatives may have aided Bacon, F. T. (1979). Credibility of repeated statements: Memory for trivia.
later performance via several different mechanisms. Reading the Journal of Experimental Psychology: Human Learning & Memory, 5,
241–252.
option on the test serves as an additional study opportunity. As
Begg, I., Armour, V., & Kerr, T. (1985). On believing what we remember.
such, it may have reminded subjects of previous knowledge that Canadian Journal of Behavioral Science, 17, 199 –214.
they would not have been able to retrieve otherwise. Reading the Bjork, R. A. (1975). Retrieval as a memory modifier: An interpretation of
alternatives may also have taught subjects answers by a process of negative recency and related phenomena. In R. L. Solso (Ed.), Informa-
deduction, if they used knowledge about the other options to select tion processing and cognition (pp. 123–144). New York: Wiley.
the correct answer by a process of elimination. Or reading the Brown, A. S. (1988). Encountering misspellings and spelling performance:
 OBSERVATIONS 1159

Why wrong isn’t right. Journal of Educational Psychology, 80, 488 – conditions of transfer. Journal of Experimental Psychology, 64, 380 –
494. 387.
Brown, A. S., Schilling, H. E. H., & Hockensmith, M. L. (1999). The Rees, P. J. (1986). Do medical students learn from multiple-choice exam-
negative suggestion effect: Pondering incorrect alternatives may be inations? Medical Education, 20, 123–125.
hazardous to your knowledge. Journal of Educational Psychology, 91, Remmers, H. H., & Remmers, E. M. (1926). The negative suggestion effect
756 –764. on true–false examination questions. Journal of Educational Psychol-
Carrier, M., & Pashler, H. (1992). The influence of retrieval on retention. ogy, 17, 52–56.
Memory & Cognition, 20, 633– 642. Rickards, J. P., & Friedman, F. (1978). The encoding versus the external
Crowder, R. G. (1976). Principles of learning and memory. Oxford, storage hypothesis in note taking. Contemporary Educational Psychol-
England: Erlbaum. ogy, 3, 136 –143.
Glover, J. A. (1989). The “testing” phenomenon: Not gone but nearly Roediger, H. L., III, Jacoby, J. D., & McDermott, K. B. (1996). Misinfor-
forgotten. Journal of Educational Psychology, 81, 392–399. mation effects in recall: Creating false memories through repeated
Hasher, L., Goldstein, D., & Toppino, T. (1977). Frequency and the retrieval. Journal of Memory and Language, 35, 300 –318.
conference of referential validity. Journal of Verbal Learning and Ver- Roediger, H. L., III, & Karpicke, J. D. (in press). Test-enhanced learning:
bal Behavior, 16, 107–112. Taking memory tests improves long-term retention. Psychological Sci-
Hogan, R. M., & Kintsch, W. (1971). Differential effects of study and test ence.
trials on long-term recognition and recall. Journal of Verbal Learning Roediger, H. L., III, Wheeler, M. A., & Rajaram, S. (1993). Remembering,
and Verbal Behavior, 10, 562–567. knowing, and reconstructing the past. In D. L. Medin (Ed.), The psy-
Izawa, C. (1970). Optimal potentiating effects and forgetting prevention chology of learning and motivation: Advances in research and theory
effects of tests in paired-associate learning. Journal of Experimental (pp. 97–134). San Diego, CA: Academic Press.
Psychology, 83, 340 –344. Runquist, W. N. (1986). The effect of testing on the forgetting of related
Jacoby, L. L., & Hollingshead, A. (1990). Reading student essays may be and unrelated associates. Canadian Journal of Psychology, 40, 65–76.
hazardous to your spelling: Effects of reading incorrectly and correctly Schooler, J. W., Foster, R. A., & Loftus, E. F. (1988). Some deleterious
spelled words. Canadian Journal of Psychology, 44, 345–358. consequences of the act of recollection. Memory & Cognition, 16,
Kulhavey, R. W., Dyer, J. W., & Silver, L. (1975). The effects of noteta- 243–251.
king and test expectancy on the learning of text material. Journal of Spitzer, H. F. (1939). Studies in retention. Journal of Educational Psy-
Educational Research, 68, 363–365. chology, 30, 641– 656.
Kuo, T.-M., & Hirshman, E. (1996). Investigations of the testing effect. Thompson, C. P., Wenger, S. K., & Bartling, C. A. (1978). How recall
American Journal of Psychology, 109, 451– 464. facilitates subsequent recall: A reappraisal. Journal of Experimental
Lewis, C. H., & Anderson, J. R. (1976). Interference with real world Psychology: Human Learning & Memory, 4, 210 –221.
knowledge. Cognitive Psychology, 8, 311–335. Toppino, T. C., & Brochin, H. A. (1989). Learning from tests: The case of
Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile true–false examinations. Journal of Educational Research, 83, 119 –124.
destruction: An example of the interaction between language and mem- Toppino, T. C., & Luipersbeck, S. M. (1993). Generality of the negative
ory. Journal of Verbal Learning and Verbal Behavior, 13, 585–589. suggestion effect in objective tests. Journal of Educational Psychology,
McDaniel, M. A., & Fisher, R. P. (1991). Tests and test feedback as 86, 357–362.
learning sources. Contemporary Educational Psychology, 16, 192–201. Tulving, E. (1967). The effects of presentation and recall of material in
McDaniel, M. A., & Masson, M. E. (1985). Altering memory representa- free-recall learning. Journal of Verbal Learning and Verbal Behavior, 6,
tions through retrieval. Journal of Experimental Psychology: Learning, 175–184.
Memory, and Cognition, 11, 371–385. Zaragoza, M. S., Payment, K. E., Ackil, J. K., Drivdahl, S. B., & Beck, M.
McGeoch, J. A. (1932). Forgetting and the law of disuse. Psychological (2001). Interviewing witnesses: Forced confabulation and confirmatory
Review, 39, 352–370. feedback increase false memory. Psychological Science, 12, 473– 477.
Meade, M. L., & Roediger, H. L., III. (in press). The effect of forced recall
on illusory recollection in younger and older adults. American Journal of Received July 26, 2004
Psychology. Revision received March 14, 2005
Postman, L. (1962). Retention of first-list associations as a function of the Accepted March 16, 2005 䡲