Original article

doi: 10.1111/jcal.12050

bs_bs_banner

Adaptive formative feedback to improve

strategic search decisions in
task-oriented reading
A.C. Llorens, R. Cerdán & E. Vidal-Abarca
Department of Developmental and Educational Psychology, University of Valencia, Spain

Abstract This study analyses the effectiveness of adaptive formative feedback to boost strategic search
decisions and performance when students are asked to answer a set of questions in a task-
oriented reading situation. We compared automatic feedback that included information about
the right answer with feedback that also included the connection between the students’
strategic search decisions and their performance. Ninety-two high school students read two
non-continuous texts. They received feedback during a training phase, and then they read and
also received feedback with a similar text in a final phase. Text and questions were presented
using a new computer-based technology that provided automatic adaptive feedback depending
on the experimental condition: right-answer feedback, strategic-search-decisions feedback
and placebo feedback. We found that strategic-search-decisions feedback improved strategic
decisions over right-answer and placebo feedback in the final text, which in turn improved
question-answering performance. Some positive effects were also found during training.
These results open new possibilities to adaptive automatic procedures to teach task-oriented
reading skills to students.

Keywords comprehension, computer-based feedback, formative feedback, specific feedback

information, strategic search decisions, task-oriented reading.

In instructional settings, students are frequently asked tions from an available text) leads students to make a
to answer a set of questions having the text available number of decisions before answering, requiring self-
for either learning or assessment purposes, which has regulation on the part of the student (i.e., Vidal-Abarca
been called task-oriented reading (Vidal-Abarca et al., et al., 2010). Specifically, in task-oriented reading
2010). These reading situations emphasize the interac- situations, students have to mainly self-regulate when
tion between the student and the text on a task that is and what information to search, which involves stra-
meaningful for the student, as recent definitions of tegic decisions to identify, and select relevant text
reading literacy point out (Snow & RAND Reading information according to the specificities of the ques-
Study Group, 2002; Organization for Economic tion they are currently answering (i.e., McCrudden &
Co-operation and Development (OECD), 2010). Task- Schraw, 2007; Rouet, 2006).
oriented reading (i.e., answering comprehension ques- Consider the following example based on a text pre-
sented in the current study. A student has to use a
non-continuous text that includes a diagram describing
Accepted: 20 October 2013
the cloning procedure (see Figure 1) to answer a
Correspondence: Ana Cristina Llorens, Department of Developmen-
tal and Educational Psychology, University of Valencia, Avd. Blasco number of multiple-choice questions, such as the next
Ibáñez 21, Valencia 46010, Spain. Email: a.cristina.llorens@uv.es one: ‘what parts of a living being are necessary to

© 2014 John Wiley & Sons Ltd Journal of Computer Assisted Learning (2014), 30, 233–251 233
234 A.C. Llorens et al.

Figure 1 Non-Continuous Text Presented in the Current Study that Includes a Diagram Describing the Cloning Procedure

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 235

develop an embryo by cloning?’ Imagine that the tegic decisions (i.e., when and what to search)? In other
student has already read the text, either completely or words, would students benefit from getting feedback
partially. To answer the question, the student must self- about their strategic search decisions in a specific ques-
regulate his task-oriented reading behaviour. This self- tion to answer further questions in a task-oriented
regulation may require first deciding when to refer back reading situation? We have precisely designed this
to the text, which may depend on monitoring the study to answer these questions. With this purpose in
current mental representation of the text (e.g., assess- mind, we have recorded the online students’ behav-
ing whether he knows the answer, or alternatively he iours regarding strategic decisions about when and
needs searching the text), and in case he decides what information to search to answer questions from a
searching, to search what text information is relevant to text, and we have given formative feedback associated
answer the question (e.g., locating the relevant text to the students’ performance. As far as we know, no
information where the parts to form an embryo by previous study has recorded the students’ online behav-
cloning are described and to discard irrelevant ones). If iours to provide this type of formative feedback aimed
the student does not search the text because he wrongly at improving strategic decisions on a task-oriented
thinks he knows the answer, or he searches non- reading scenario. We assume that this kind of feedback
relevant information, he would probably give a wrong might help students to modify their strategic decisions,
answer. Thus, self-regulating when and what informa- which may in turn increase performance.
tion to search is a crucial decision in a task-oriented To better set the goals for this study, we will first
reading scenario. clarify how students self-regulate the search for infor-
Unfortunately, research has shown that in task- mation in task-oriented reading activities which entails
oriented reading, many students self-regulate their strategic search decisions; second, we will explain in
when and what decisions quite poorly. Indeed, detail how formative feedback can be an effective tool
Vidal-Abarca et al. (2010) found that students decided to foster students’ strategic search decisions when they
to search the text fewer times than they would need, answer questions from an available text. Based on this
even though searching for information can be helpful. review, we will set the hypothesis and predictions for
Consequently, they give wrong answers when they the present study.
might have responded right (i.e., Mañá, 2011;
Metacalfe & Finn, 2008; Vidal-Abarca et al., 2010).
Search decisions in task-oriented reading
Furthermore, risky no-search decisions may appear
more frequently when students are confronted with Task-oriented reading situations, such as answering
non-continuous texts like in the initial example comprehension questions from an available text, have
(Figure 1), as the integration of iconic and verbal infor- two essential characteristics. First, not all the informa-
mation is a highly demanding task (Schnotz, 2005). tion within the text is equally relevant for the reader to
Actually, answering questions from texts that include answer each question. The specificities of the task (e.g.,
verbal and iconic information increased the need of to answer questions from a text) act as guidelines for
searching compared with reading texts that include the text processing and help the readers to focus on
only textual information (i.e., Serrano, Vidal-Abarca, the relevant text information to perform the task
& Ferrer, submitted). (McCrudden, Magliano, & Schraw, 2010; McCrudden
At this point, imagine that the student is answering & Schraw, 2007). Second, the readers interact with the
questions through a computer-based system that would text in a particular way, going back and forth from
automatically provide formative feedback which the text to the task, and vice versa, until they consider
allows him to compare his current performance with the task is performed correctly (Vidal-Abarca et al.,
some desired standard of performance (i.e., Hattie & 2010). For instance, students would stop searching
Timperley, 2007; Mory, 2004; Shute, 2008). Would it when they decide they have enough information to
be enough to inform students about the correctness of provide a right answer to a question. Therefore, self-
their answers and show the right choice to a specific regulation in task-oriented reading requires compre-
multiple-choice question or, alternatively, would they hension not only of the text but also of the task. Indeed,
benefit from an additional information about their stra- current research has demonstrated that task-oriented

© 2014 John Wiley & Sons Ltd

236 A.C. Llorens et al.

reading puts specific demands on the reader, Anderson, & Therriault, 2003; Thiede, Griffin, Wiley,
such as self-regulating the search for information, & Redford, 2009). Furthermore, in task-oriented
beyond classical comprehension processes (e.g., reading situations, students were overconfident evalu-
Cataldo & Oakhill, 2000; Maña, Vidal-Abarca, ating their understanding, and they decided not to
Domínguez, Gil, & Cerdán, 2009; Vidal-Abarca et al., search the text although this decision led them to fail
2010). more than succeed in answering questions (Mañá,
Theories of comprehension (e.g., Graesser, Singer, 2011; Vidal-Abarca et al., 2010). Consequently, task-
& Trabasso, 1994; Kintsch, 1998; van den Broek, oriented reading performance improved when stu-
Young, Yuhtsuen, & Linderholm, 1999) have concen- dents increased the number of their search decisions
trated on how readers comprehend a text, but they have (Mañá, 2011; Vidal-Abarca, Salmerón, et al., 2011).
not explicitly addressed the strategic search decisions Previous research has shown that students improved
inherent to task-oriented reading situations (i.e., their comprehension scores when they were directly
McNamara & Magliano, 2009). Rouet’s (2006) Text- forced to search the text before answering a specific
based Relevance Assessment and Content Extraction question (i.e., Mañá, 2011); or when they were
(TRACE) model is a general framework which required to answer questions after a delay between
describes the specific text–reader interaction which reading the questions and providing an answer,
entails task-oriented reading situations. The TRACE which in turn led them to increase their number of
model places the emphasis on the decisions made by search decisions (i.e., Vidal-Abarca, Salmerón, et al.,
the reader as regards the need to search for information 2011).
or not (i.e., decision about when to refer back to the Furthermore, once students have decided to search
text) and the selection of information that matches the the text, the search process involves a selective reading
task demands (i.e., decisions regarding what text infor- in which the readers must decide what text information
mation is relevant to answer the question). Both deci- is relevant for their purposes and ignore the rest (Rouet
sions are crucial elements in self-regulating the & Vidal-Abarca, 2002). When searching, students have
strategic search behaviour in task-oriented reading to understand incoming information while they bear in
situations. Furthermore, recent studies have shown mind the mental model of the question demands. In
that students who made strategic search decisions to addition, students must perform a relevance-based
find relevant text information had better comprehen- strategy that entails locating relevant text parts and
sion scores in task-oriented reading than those discarding irrelevant ones. Different studies have
who decided not to search or searched non-relevant shown that a search process not directed to relevant
information (e.g., Cerdán, Gilabert, & Vidal-Abarca, information to perform the task leads to poor
2011; Vidal-Abarca et al., 2010). Consequently, self- reading literacy scores (e.g., Cataldo & Oakhill, 2000;
regulating strategic search decisions in task-oriented Cerdán & Vidal-Abarca, 2008; Cerdán et al., 2011;
reading situations seem to be essential for success in Vidal-Abarca et al., 2010).
these kinds of tasks, which are the focus of the present Thus, it can be stated that strategic search decisions
paper. (i.e., when and what) are at the core of task-oriented
Specifically, the self-regulation of search decisions reading situations. Thus, providing students with feed-
(i.e., whether to search for text information or not) back aimed at increasing their search decisions is sup-
starts with a monitoring process of one’s mental rep- posed to be beneficial for performance. Actually, Ávila,
resentation of the text. Based on this first calibration, Vidal-Abarca, Gil, and Llorens (2010) and Llorens,
a search is undertaken or not. Just when students do Vidal-Abarca, Cerdán, and Ávila (submitted) have pro-
not believe they know the answer they may decide to vided evidence about the effectiveness of formative
search, that is, they self-regulate their search behav- feedback as an instructional procedure to boost an
iours (i.e., Vidal-Abarca et al., 2010). Overall, stu- accurate search behaviour based on relevant text infor-
dents’ monitoring accuracy has typically been quite mation that, in turn, improves performance in task-
low when they were required to read texts and make oriented reading situations. Specifically, students
judgments of their performance on future tests answered multiple-choice questions and then they
(Dunlosky & Lipko, 2007; Maki, 1998; Thiede, received feedback about their choices (i.e., right-wrong

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 237

feedback). If the choice was wrong, then they received

Formative feedback to foster students’ strategic
additional formative feedback, and then they had a
search decisions
second chance to give an answer. The feedback was
either search-decisions feedback (i.e., a general advice The main goal of formative feedback is to increase the
to encourage students to search, such as ‘try to search; students’ knowledge in a specific content area or to
it would be helpful’), or relevant-search-decisions develop a general skill (e.g., making strategic deci-
feedback (i.e., which provides the location of the rel- sions in task-oriented reading). Formative feedback
evant text information to answer the question, such as allows students to compare their performance in a
‘re-reading in the page X the information about “XX given task with some desired standard of performance
XX XX” will help you to answer correctly’). Both types (i.e., Hattie & Timperley, 2007; Mory, 2004; Shute,
of feedback made students focus on the particular ques- 2008). Generally speaking, formative feedback has
tion they were currently answering in order to correct been considered significantly more effective when
their initial wrong answers. Relevant-search-decisions it provides details of how to improve the answer
feedback was more effective to avoid monitoring inac- (i.e., Hattie & Gan, 2011; Mason & Bruning, 2001;
curacies than search-decisions feedback. In other Narciss, 2004; Shute, 2008). Thus, an important issue
words, students in the first condition followed the feed- regarding the formative feedback messages, focused
back recommendations more often than those in the on either content or skills learning, is how much infor-
second condition, and thus they increased their number mation should be included in the message. In other
of search decisions. Then, they employed the relevant words, apart from informing students whether they
text information to improve their success in correcting fail or succeed, what other type of information should
wrong answers (e.g., Rawson & Dunlosky, 2007). Con- we include in the feedback?
sequently, in task-oriented reading, not only deciding Previous research has shown that providing students
to search is a key process but determining what infor- with the right answer to a specific question was more
mation is relevant for a task. effective than simple failure vs. succeed feedback (e.g.,
To sum up, two strategic decisions are essential to Bangert-Drowns, Kulik, Kulik, & Morgan, 1991;
answer questions from a text in task-oriented reading Pashler, Cepeda, Wixted, & Rohrer, 2005; Phye &
situations, that is, deciding when to refer back to Sanders, 1994). Thus, it might be assumed that inform-
the text and what text information is relevant to ing students about the right answer after failure would
answer the question. These two decisions involve boost students’ strategic search decisions (i.e., when
self-regulation on the part of the student which is and what) as students might reflect on their behaviours
quite often problematic (e.g., Cerdán et al., 2011; and change their search strategies. Applying this rea-
Vidal-Abarca et al., 2010). Giving learners formative soning to the example given at the beginning of this
feedback on where the relevant text information was paper, after giving a wrong answer the student would
located in the text had a positive impact on answering receive a message informing him about the failure plus
the questions. However, the effectiveness of this feed- information about the right answer (i.e., right-answer
back was confined to the particular question where the feedback), which might lead him to modify his stra-
feedback was delivered (Ávila et al., 2010; Llorens tegic search decisions. For instance, if he did not refer
et al., submitted). As a consequence, students did not back to the text or he did not find relevant information,
improve their strategic search decisions in further then he might decide to refer back to the text or to
questions. That is, the strategies learned were not invest a greater effort to find relevant information in
transferred to different questions. In the current study, order to achieve the right answer in subsequent ques-
we implemented a formative feedback procedure that tions. Therefore, providing right-answer feedback
would induce students to change their strategic search might act as a cue to identify the desired performance,
decisions (i.e., when and what) in answering subse- which in turn would improve when and what strategic
quent questions, which in turn would increase their decisions.
performance in further questions. We explore the However, a right-answer feedback might not be
possible mechanism of this feedback in the next enough to improve the students’ strategic search deci-
section. sions as it would provide no specific information on

© 2014 John Wiley & Sons Ltd

238 A.C. Llorens et al.

how to improve strategic search decisions in further

Current study
questions. In fact, some studies have supported the idea
that presenting the right answer with additional rel- The goal of the present study was to compare the effi-
evant information to perform the task increased learn- cacy of the two types of formative feedback explained
ing over just informing learners about the right answer previously in order to boost strategic search decisions
(e.g., Butler, Godbole, & Marsh, 2013; Whyte, and performance in a task-oriented reading situation.
Karolick, Neilsen, Elder, & Hawley, 1995). In task- When students answered a multiple-choice question,
oriented reading, this additional information should they received either right-answer feedback or strategic-
focus on the strategic when and what decisions, as search-decisions feedback. Both feedback messages
in these reading situations comprehension scores included information about failure or success as well as
improved when students increased the number of when information about the right answer. However, strategic-
decisions (e.g., Mañá, 2011; Vidal-Abarca, Salmerón, search-decisions feedback also informed students about
et al., 2011) and when students were able to identify their strategic when and what decisions and a recom-
what text information is relevant to answer correctly mendation to act strategically in subsequent questions.
(i.e., Ávila et al., 2010; Cerdán et al., 2011; Llorens We also included a control condition where students
et al., submitted). Furthermore, this formative feed- received placebo feedback (i.e., neutral messages about
back should adapt to the students’ initial strategies (i.e., the order of the questions). Feedback messages were
Hattie & Timperley, 2007). Applying this reasoning to delivered through a new computer-based technology
the initial example, the additional information deliv- developed for this study that keeps track of the students’
ered through a computer-based system should connect strategic search decisions (i.e., when and what) when
the end result of performance with the students’ when interacting with the text and the questions and provides
and what strategic decisions (i.e., strategic-search- automatic feedback for the experimental conditions.
decisions feedback). For instance, the student would As the information provided in strategic-search-
receive a feedback message informing him about the decisions feedback was individualized, and it depended
connection between his current performance and on the students’ when and what decisions, we expected
search decisions (e.g., You failed! When you have a huge variability in the messages students would
searched the text, you have not reread information receive. In other words, some students would fail and
which is necessary to answer), as well as a specific search very little, whereas others would do exactly the
recommendation on how to proceed in subsequent opposite, with some others in-between. Given that stu-
questions (e.g., In the following questions, rereading dents would need extensive practice to learn the connec-
the necessary information will help you to correctly tion between search decisions and performance
answer). Then, the student would also know the right (Bransford, Brown, & Cocking, 2000), we made stu-
answer to the question. This alternative feedback might dents practice with two texts in a training phase, and
be more helpful than the right-answer feedback as it is then we presented a text in a final phase that also
more explicit and informative. It would also provide included feedback messages to examine the change in
the student with an opportunity to confirm or restruc- their strategies and the corresponding performance
ture his search decisions when answering further ques- effect. In this way, we also increased the likelihood that
tions (i.e., Butler & Winne, 1995). Please note that this a student would experience different feedback messages
feedback would connect performance and strategic that included failure or success information, and very
decisions regardless of success or failure in the ques- different behaviours regarding when and what strategic
tion. That is, when the student’s answer is right and he decisions, in the case of students in the strategic-search-
has also searched the right information, then he would decisions feedback condition. It should be noted that all
also receive a message connecting strategic search messages in this latter condition emphasized the same
decisions and performance (e.g., Right! During the strategies, which are the convenience of searching the
search you have read information which is necessary text when answering, and the need to focus on relevant
to answer correctly. This way to answer has been information when searching. Moreover, these strategies
effective. It will certainly help you to answer other were emphasized regardless of failure or success. We
questions). assumed that the presence of feedback in the final phase

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 239

would act as a prompting cue to ensure the implemen- were Spanish native speakers) participated in the study.
tation of strategic search decisions that would lead They were randomly distributed into three groups
students to better self-regulate their search behaviours according to the formative feedback type. The final
(e.g., Gick & Holyoak, 1980; Perfetto, Bransford, & sample distribution was as follows: there were 30 stu-
Franks, 1983) and improve performance. dents in the strategic-search-decisions feedback group,
Therefore, the effect of formative feedback on the 34 students participated in the right-answer feedback
students’ strategic search decisions and performance group, and 28 students participated in the placebo feed-
should appear clearly in the final phase, although it back group. Students were assigned to each condition
might also be apparent in the training phase as a result after being tested with a standardized comprehension
of the students’ practice processes. Thus, the increase test (CompLEC; Llorens et al., 2011) so that all groups
of strategies regarding search decisions and the use of were similar in terms of comprehension skills, F(2,
relevant information would be indicative of formative 89) = 0.13, p = .88 (M = 12.36, sd = 3.77 for the
feedback effectiveness. This increase would also placebo group; M = 12.82, sd = 3.61 for the right-
produce better performance on the questions (e.g., answer group; M = 12.57, sd = 3.57 for the strategic-
Mañá, 2011; Vidal-Abarca, Salmerón, et al., 2011). search-decisions group).
According to the ideas presented previously, we pre-
dicted that strategic-search-decisions feedback would
Materials
help students in the final phase (1) to increase the
number of search decisions (i.e., when decisions) and Apparatus
(2) to improve the use of relevant information (i.e., The texts and questions were presented using a new
what decisions) in a greater extent than right-answer computer-based technology based on Read&Answer
feedback and placebo feedback. It should be noted that software (Vidal-Abarca, Martínez et al., 2011). The
strategic-search-decisions feedback provides students program recorded every action the students undertook,
with specific information on how to improve their stra- the order of these activities and the time each activity
tegic search decisions in further questions which lasted (i.e., online processing measures) using
should be more effective than just informing about the masking/unmasking procedure. The students had to
right answer (e.g., Butler et al., 2013; Whyte et al., unmask text segments by clicking on them (Figure 2).
1995). Furthermore, we expected no differences in the In addition, they had to click on both the question and
final phase between right-answer feedback and the answer options that remained masked in order to
placebo feedback as neither of them provide informa- read it (Figure 3). A simple interface allowed the
tion that guides students’ search decisions. Finally, the readers to move from the text screen to the questions
predicted positive impact of strategic-search-decisions screen and from one question to another. The software
feedback on students’ when and what decisions would allowed us to display formative feedback messages
increase students’ performance on the questions as according to the feedback group once students marked
both strategic search decisions would foster higher their answer choices (Figure 4). In addition, when stu-
comprehension scores in task-oriented reading (e.g., dents closed the feedback message the system showed
Cerdán et al., 2011; Mañá, 2011; Vidal-Abarca et al., the students the choices marked with either a ‘green tic’
2010; Vidal-Abarca, Salmerón, et al., 2011). Thus, we for the right answer and ‘red crosses’ for the incorrect
predicted that strategic-search-decisions feedback par- choices (Figure 5). The main advantage of this new
ticipants would obtain better performance scores in the software is that it allows to display formative feedback
final phase than right-answer feedback and placebo automatically based on the recording of students’ when
feedback students. and what search decisions according to the experimen-
tal conditions.
Method
Previous comprehension skills
Participants
We used the Test of Reading Literacy for Secondary
Ninety-two high school students from grades 7 and 8 Education (CompLEC; Llorens et al., 2011) developed
(mean age 12.53, sd = 1.47; 67% male, all of them following the PISA framework. This standardized test

© 2014 John Wiley & Sons Ltd

240 A.C. Llorens et al.

Figure 2 Caption of the Masked/Unmasked Text Procedure and the Navigation Toolbar Displayed with New Computer-Based Tech-
nology (Access to the Question Screen by Clicking on ‘¿preguntas? [¿questions?]’)

included five texts (i.e., continuous and discontinuous) the development of new bowling machines and a
and 20 questions (i.e., maximum score was 20). The diagram with text and pictures showing the bowling
questions required retrieving, interpreting, integrating collection procedure. Finally, ‘Cloning’ showed text
or reflecting beyond the text information as defined by information about the origin, procedure and types of
PISA framework. cloning with a table and a diagram which illustrated the
cloning development and cloning procedure, respec-
Texts tively. Answering questions from the texts highly
We employed three non-continuous texts (i.e., OECD, required that students search for relevant information.
2010) that combined textual and iconic information
(i.e., Ski trip, Bowling Machine and Cloning) with a Questions
length of approximately 400 words each. All texts were We employed eight multiple-choice comprehension
divided into two pages, which contained a similar questions per text (i.e., maximum score per text was 8).
number of words. The text entitled ‘Ski trip’ took the We constructed two different types of questions from
form of a brochure and included information about the PISA framework: there were 50% of questions
requirements, places, dates and scheduling for young prompting the students to access and retrieve specific
people who want to ski. In addition, the text presented units of information and questions that required stu-
a table with prices and discounts for different services. dents to interpret or integrate textual information.
‘Bowling Machine’ presented text information about Please note that all questions required comprehension,

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 241

Figure 3 Caption of the Masked/Unmasked for the Question (Upper) and the Answer Options (Lower) in the New Computer-Based
Technology. The Text in Miniature (Left Side) Allows Returning to the Text Screen

although the amount of inferential activity varied centage of performance success in the questions of
among questions. each text) and the students’ search behaviours (i.e.,
percentage of questions in which students searched the
Pilot study and experimental procedure
text to answer a question) for each text in order to
We performed a pilot study with an independent establish a specific order of the text presentation. The
sample of 32 students to test the difficulty (i.e., per- objective was to present two training texts (i.e., to

© 2014 John Wiley & Sons Ltd

242 A.C. Llorens et al.

Figure 4 Captions of the Formative Feedback Messages that Varied According to the Feedback Group

ensure that students had sufficient opportunities to questions. We presented captions of the procedure
learn the strategies fostered by each type of feedback) using a short text with one question as example. Then,
and a final text where students were expected to have we asked students to read fully the text before access-
improved their strategic decision skills and perfor- ing the questions in order to reduce the variability
mance. Then, we decided to present the two easiest among students in reading and search strategies due to
texts first as training texts (.52 and .53 for Ski trip and individual differences in the amount of text read ini-
Bowling Machine, respectively) and the most difficult tially. After reading, they were told to read and answer
text as a final text to maximize the effect of training the questions linearly, being allowed to refer back to
(i.e., .40 for Cloning). Search behaviour, that is, the the text to answer the questions at their will. Students
number of questions where students decided to search were also informed that they would receive a feedback
to give an answer, was equivalent among texts (.58 for message after every question they had to read carefully.
Ski trip; .55 for Bowling Machine, .60 for Cloning). Next, we explain the experimental conditions in detail.
The experiment lasted two sessions of approxi-
mately 1 h. In the first session, we tested participants
Experimental conditions
on reading literacy skills (i.e., CompLEC). In the
experimental session, students first read the two texts Right-answer feedback
for the training phase (i.e., training texts), and then the Students in this condition received formative feedback
text for the final phase. Students were first instructed on messages that included the correctness of their answers
how to use the software to read the texts and answer the (i.e., right or you failed). When they closed the

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 243

Figure 5 Captions of the Answer Options after Reading the Feedback Messages with either a ‘Green Tic’ for the Right Answer and ‘Red
Crosses’ for the Incorrect Choices

message, the system showed the students the choices proceed to further questions that linked search decision
marked with either a ‘green tic’ for the right answer and and performance. More specifically, when students
‘red crosses’ for the incorrect choices (Figure 5). gave a wrong answer and they did not make the right
strategic decisions regarding when and what to search,
Strategic-search-decisions feedback the feedback message provided an explicit instruction
Students in this condition received the same right- (e.g., In the following questions, read carefully the
answer feedback explained previously plus additional information which is necessary to answer correctly).
information regarding the students’ strategic search However, when students gave a right answer but they
decisions. In this way, we tried to emphasize the asso- did not make the right strategic decisions, the message
ciation between performance and students’ strategic provided an advice (e.g., In the following questions,
decisions. Therefore, besides the information about the think if you should search the text to ensure a correct
correctness of their answers (i.e., right or you failed), answer). Finally, when students gave a right answer
students in this condition were informed about their and made the right strategic decisions, the message
strategic search decisions about when to search (e.g., highlighted their right strategic decisions (e.g., During
You have not searched the text to answer) and/or what the search you have read information which is neces-
information they had searched (e.g., When you have sary to answer correctly. This way to answer has been
searched the text, you have not reread information effective. It will certainly help you to answer other
which is necessary to answer). The messages also questions). Please note that we always emphasized the
included an explicit recommendation on how to connection between the appropriate search decision

© 2014 John Wiley & Sons Ltd

244 A.C. Llorens et al.

regarding when and what to search and performance so Performance measures

that students had many opportunities to set this con- Performance measures were calculated according to
nection. Finally, when they closed the initial message, the students’ literacy scores (i.e., questions scores).
the program showed the question marking the right Thus, we calculated (1) the percentage of performance
answer by a ‘green tic’ and the incorrect choices by success in the final phase as the number of right
‘red crosses’ (Figure 5), as it happened in the previous answers each student had obtained in this text divided
experimental condition. by the maximum number of points that each student
could obtain (eight points). Further, we calculated (2)
Placebo feedback the percentage of performance success in the training
This condition included neutral feedback about the texts. It was computed as the number of right answers
order of the questions (e.g., You have answered ques- each student had obtained in both training texts divided
tion number 5). The goal was to interrupt the answering by the maximum number of points that each student
process with a message as we did in the other experi- could obtain (16 points).
mental conditions. It should be noted that online and performance
measures were directly comparable in the training and
final phases because all the variables referred to stra-
Measurement tegic search decisions were calculated on the basis of
one text (final phase) and on the average between the
A number of online measures were considered using a
two tests (training phase).
new computer-based technology in order to analyse
students’ strategic search decisions. These online pro-
cessing measures focused on when to refer back to the Results
text and what text information is relevant to answer the
To test our hypothesis, we performed a set of analyses
question. In addition, we computed the reading literacy
of covariance with independent variable type of
scores (i.e., performance) in the three experimental
feedback (right-answer feedback, strategic-search-
conditions.
decisions feedback or placebo feedback), comprehen-
sion skills as covariable, and online processing
Online processing measures measures and performance scores as dependent
variables.
When decisions
To calculate this online variable, we computed (a) the Effect of formative feedback on strategic search
total number of search decisions or number of times decisions in the final phase
students referred back to the text to answer a question We predicted that strategic-search-decisions feedback
in the final phase of the study (i.e., dichotomous vari- would increase students’ number of search decisions
able). Additionally, we calculated (b) the average and their use of relevant information (i.e., when and
search decisions per text in the training phase. what decisions) in the final phase over the remaining
experimental groups. To test this prediction, we ana-
What decisions lysed each online processing measure in the final
We first computed a measure of the use of relevant phase, where students had already practiced with the
information before answering to analyse the search feedback recommendation in the previous training
process. Thus, we calculated (1) the number of times phase.
that students found relevant information just before Regarding when decisions, there was a significant
answering a question in the final phase. In addition, we main effect for the number of students’ search decisions
computed (2) the average number of times that students in the final phase, F(2, 88) = 6.90, p = .00, partial
found relevant information just before answering a η2 = .14. Students in strategic-search-decisions group
question in the training texts. Finally, we computed (1) overcame students in right-answer and placebo feed-
the average time spent searching the text per question back groups (B = − 0.25, SEB = 0.11, t(91) = − 2.22,
in the final text, as well as (2) in the training texts. p = .03 and B = − 0.46, SEB = 0.13, t(91) = − 3.66,

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 245

p = .00, respectively). Right-answer and placebo feed- feedback were unable to improve their strategic search
back groups did not differ from each other, B = − 0.19, decisions over placebo feedback.
SEB = 0.13, t(91) = − 1.44, p = .15. Means and standard
deviations are summarized in Table 1. No interactions
Effect of formative feedback on strategic search
between any experimental group and comprehension
decisions in the training phase
skills were found for the number of times students
referred back to the text. Although there was no specific prediction regarding the
Regarding what decisions, a significant main effect effect of feedback on the training phase, we could
was found for the number of times that students used expect that the effect of strategic-search-decisions
relevant information in the final phase, F(2, 88) = 3.73, feedback in the final phase (i.e., increasing when
p = .03, partial η2 = .08. Students in strategic-search- and what decisions over the remaining experimental
decisions overcame right-answer and placebo feed- groups) might also be apparent in the training phase as
back groups (B = − 0.22, SEB = 0.11, t(91) = − 2.12, a result of the students’ practice processes. Hence, we
p = .04 and B = −0.31, SEB = 0.09, t(91) = − 2.55, explored equivalent online processing measures in the
p = .01, respectively). Right-answer and placebo feed- training phase (i.e., calculated as the average between
back groups did not differ from each other, B = − 0.07, the two tests) to those in the final phase (i.e., on the
SEB = 0.13, t(91) = − 0.53, p = .60 (see Table 1). No basis of one text).
interactions with comprehension skills were found. Regarding when decisions, we found a significant
Thus, the strategic-search-decisions group used rel- main effect of type of formative feedback for the
evant information more times to answer the questions number of search decisions in the training phase,
than the remaining groups in the final phase. Moreover, F(2, 88) = 3.98, p = .02, partial η2 = .08. Post hoc tests
a significant main effect was also found for the average revealed that strategic-search-decisions group over-
time searching the text per question, F(2, 88) = 10.62, came placebo group, B = − 0.36, SEB = 0.13, t(91) =
p = .00, partial η2 = .19. Students in strategic-search- − 2.78, p = .01. Right-answer feedback did not differ
decisions overcame right-answer and placebo groups, from strategic-search-decisions and placebo groups
B = − 0.34, SEB = 0.12, t(91) = − 2.90, p = .01 and (B = − 0.13, SEB = 0.12, t(91) = − 1.08, p = .28 and
B = − 0.57, SEB = 0.13, t(91) = − 4.48, p = .00, respec- B = − 0.23, SEB = 0.10, t(91) = − 1.74, p = .09,
tively. Right-answer and placebo did not differ from respectively). Means and standard deviations are sum-
each other, B = − 0.20, SEB = 0.14, t(91) = − 1.43, marized in Table 2. No interactions between any
p = .16 (see Table 1). No interaction with comprehen- experimental group and comprehension skills were
sion skills was found for the time searching the text per found for the number of times students referred back
question. to the text in the training phase. Thus, strategic-
In sum, according to our predictions in the final search-decisions already outperformed placebo for the
phase, the formative feedback that provides informa- number of search decisions in the training phase. In
tion on how to improve strategic search decisions in addition, although the differences between strategic-
subsequent questions (i.e., strategic-search-decisions) search-decisions and right-answer did not achieve
was superior for improving students’ strategic search significance, they were in the same direction in the
decisions (i.e., when and what) than just providing the final phase where the effect of feedback appeared
right answer. By contrast, students in right-answer clearly.

Table 1. Means and Standard Deviations (in Parenthesis) of Online Processing Data in the Final Phase by Type of Formative Feedback

Placebo Right-answer Strategic-search-decisions

feedback feedback feedback

Search decisions 3.32 (2.26) 4.26 (2.47) 5.47 (2.10)

Use of relevant information 2.14 (1.76) 2.47 (1.85) 3.27 (1.82)
Searching time per question 6.75 (5.69) 9.94 (7.73) 16.11 (9.55)

© 2014 John Wiley & Sons Ltd

246 A.C. Llorens et al.

Table 2. Means and Standard Deviations (in Parenthesis) of Online Processing Data in the Training Phase by Type of Formative Feedback

Placebo Right-answer Strategic-search-decisions

feedback feedback feedback

Search decisions 4.55 (2.26) 5.43 (1.78) 5.90 (1.44)

Use of relevant information 3.98 (2.06) 4.22 (1.61) 5.30 (1.55)
Searching time per question 17.82 (11.71) 20.04 (9.37) 22.72 (8.87)

Regarding what decisions, in the training phase we students’ performance success according to the type of
found a significant main effect of feedback type for the feedback in order to further explore how improved
number of times that students used relevant informa- strategic search decisions would increase students’
tion, F(2, 88) = 3.09, p = .05, partial η2 = .07. Students performance.
in strategic-search-decisions overcame right-answer,
B = − 0.25, SEB = 0.11, t(91) = − 2.34, p = .02, and
Effect of formative feedback on performance
marginally overcame placebo feedback, B = − 0.22,
SEB = 0.12, t(91) = − 1.81, p = .07. Right-answer and To test whether strategic-search-decisions feedback
placebo feedback groups did not differ from each would increase students’ performance relative to the
other, B = − 0.05, SEB = 0.13, t(91) = − 0.36, p = .72 remaining experimental groups, we compared how
(Table 2). No interactions with comprehension skills they differed in the percentage of performance success
were found. As well as in the final phase, the effect of in the final phase. According to our predictions, we
feedback on the use of relevant information already found a significant main effect of type of formative
appeared in the training phase. However, in the training feedback for the percentage of final text success, F(2,
texts, students from all groups spent a similar average 88) = 3.19, p = .05, partial η2 = .07. Furthermore, post
time searching the text per question, F(2, 88) = 1.72, hoc analyses indicated that strategic-search-decisions
p = .19 (see Table 2). No interactions with comprehen- group outperformed placebo group, B = − 0.31, SEB =
sion skills were found. Thus, in the training phase 0.12, t(91) = − 2.52, p = .01. However, strategic-
strategic-search-decisions did not impact the students’ search-decisions did not differ from right-answer feed-
time searching the text per question as in the final back, B = − 0.15, SEB = 0.11, t(91) = − 1.34, p = .18.
phase. Right-answer feedback and placebo did not differ from
In sum, results in the training phase showed that each other, B = − 0.15, SEB = 0.13, t(91) = − 1.16,
students in strategic-search-decisions followed the p = .25. Means and standard deviations are summa-
feedback recommendation to some extent during the rized in Table 3. No interactions between experimental
practice process. Specifically, participants in strategic- conditions and comprehension skills were found for the
search-decisions used relevant information to answer percentage of final text success.
questions in a greater extent than the remaining groups, Hence, in the final phase, strategic-search-decisions
but failed to overcome significantly right-answer feed- feedback led to better performance scores than the
back for the number of search decisions. In addition, control group. In addition, right-answer feedback was
students in strategic-search-decisions spent more time not effective to lead students to a better performance
searching the text than the remaining groups in the final than placebo feedback. By contrast, the current
phase but not in the training phase. Next, we analyse findings failed to show significant differences on

Table 3. Means and Standard Deviations (in Parenthesis) of Comprehension Skills and Performance by Type of Formative Feedback

Placebo Right-answer Strategic-search-decisions

feedback feedback feedback

Percentage of final success .46 (.19) .53 (.20) .59 (.19)

Percentage of training success .59 (.19) .62 (.18) .62 (.21)

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 247

performance between strategic-search-decisions and As expected, students’ strategic decisions about

right-answer feedback in the final phase, although the when and what to search (i.e., number of search deci-
differences were in the predicted direction. It should be sions and use of relevant information) correlated with
noted that strategic-search-decisions included the performance in both phases. This finding supports the
right-answer feedback information. Thus, presenting idea that giving explicit feedback to students about the
the right answer without additional information was relationships between these decisions and performance
unable to overcome the placebo group. Strategic infor- is positive, and it contributes to explain the benefits of
mation regarding how to improve further answers was the strategic-search-decisions feedback condition.
required to overcome the placebo feedback control Only the time that student spent searching the text per
group. question seems not to be closely related to performance
In the training phase, we also compared how the in the final phase, which might be explained by the
experimental groups differed in the percentage of per- difficulty of the text used in this phase, an issue that we
formance success in order to test whether type of feed- discuss in more detail in the next section.
back would affect performance in this phase. No
significant differences were found according to the
Discussion
type of feedback for the percentage of training success,
F(2, 88) = .23, p = .79 (see Table 3). Thus, there was no The aim of the study was to test the effect of two
effect of feedback on performance success in the train- different personalized feedbacks after answering ques-
ing phase. tions (i.e., right-answer and strategic-search-decisions
In sum, the findings regarding performance in both feedback) on strategic search decisions regarding when
final and training phases suggested that students and what information to search, as well as their impact
needed several trials to implement strategic search on performance. Both feedbacks included information
decisions which lead them to a better performance, thus about the right answer, but the last one also included
providing validity to the separation between final and additional information that made the connection
training phases in this experiment. between strategic search decisions and performance
explicit. According to previous research, effective
formative feedback should provide details of how to
Correlations between strategic search decisions improve students’ performance (i.e., Hattie & Gan,
and performance 2011; Mason & Bruning, 2001; Narciss, 2004; Shute,
To provide additional evidence of the relationship 2008). The additional feedback information was
between strategic decisions and performance, we per- focused on strategic when and what decisions as crucial
formed a set of Pearson’s correlations between the elements in self-regulating text processing in task-
percentages of success in both the final phase and the oriented reading situations (Rouet, 2006). Indeed, stu-
training phase, and the corresponding measures of stu- dents who self-regulate these two strategic search
dents’ strategic search decisions in both phases. In the decisions have better comprehension scores in task-
final phase, performance correlated significantly with oriented reading (i.e., Cerdán et al., 2011; Mañá, 2011;
the number of search decisions (r = .42, p < .01) and Vidal-Abarca et al., 2010; Vidal-Abarca, Salmerón,
with the number of times that students used relevant et al., 2011). Thus, we predicted that the feedback that
information in that phase (r = .49, p < .01). By con- provided information regarding students’ strategic
trast, time searching the text per question in the final search decisions and performance (i.e., strategic-
phase did not correlate with performance in the final search-decisions feedback) would be more effective
phase, although it approached significance (r = .18, than just giving students feedback about the right
p = .07). Regarding the training phase, performance answer (i.e., right-answer feedback) or not giving a
significantly correlated with the number of search deci- real feedback (i.e., placebo feedback). Students
sions (r = .42, p < .01), with the number of times that received feedback while they were trained with two
students used relevant information (r = .51, p < .01), non-continuous texts that included iconic and verbal
and with the time searching the text per question in the information that needed integration to answer some
training phase (r = .27, p = .01). questions (i.e., training phase), and then they were

© 2014 John Wiley & Sons Ltd

248 A.C. Llorens et al.

examined with another non-continuous text in a final training phase. Thus, a certain amount of practice
phase in conditions similar to those in the training seems to be necessary to allow students an adequate
phase. We gave such training and measured the effect level of initial learning of the desired skills (i.e.,
with a final task (i.e., final phase) that also included Bransford et al., 2000). In addition, the higher diffi-
feedback messages because we assumed that students culty of the text presented in the final phase could have
would need practice and sustained feedback to consoli- made students who received strategic-search-decisions
date the appropriate strategic decisions. feedback become more aware of the need to apply
According to our predictions, providing formative strategic search decisions learned in the training phase
feedback about students’ search strategic decisions to the final text. This could be a complementary expla-
improved when and what decisions in the final phase of nation for this result.
the study relative to right-answer feedback and the Moreover, in the current study, right-answer and
control group (e.g., Butler et al., 2013; Whyte et al., placebo feedback experienced a decrement in their
1995). Thus, strategic-search-decisions feedback fos- online processing and performance values in the final
tered students’ when decisions in the final phase and phase compared with the training phase. These values
also led students to focus their search on relevant text remained similar in both phases for the strategic-
information (i.e., what decisions) to a greater extent search-decisions group. These findings suggested that
than the remaining feedback conditions. As explained feedback that guides students’ strategic search deci-
previously, both strategic search decisions were sions leads them to invest a greater effort to find rel-
expected to positively impact question scores (i.e., evant information in the final text, as revealed by the
Cerdán et al., 2011; Mañá, 2011; Vidal-Abarca et al., higher searching time of strategic-search-decisions
2010; Vidal-Abarca, Salmerón, et al., 2011). Conse- group over the other groups. These differences did not
quently, students in the strategic-search-decisions appear in the training phase. Therefore, students who
feedback group obtained better performance success in received strategic-search-decisions feedback could
the final phase than students in the control group; dif- appropriately face the challenges of a more demand-
ferences between strategic-search-decisions and right- ing task (i.e., the final phase that included the most
answer were not statistically significant, although they difficult text) by applying the strategies learned previ-
were in the predicted direction. The impact of feedback ously to answer the questions (i.e., training phase).
should be interpreted in light of the positive correlation This seemed not to occur for the other experimental
among all the dependent measurements (i.e., strategic groups.
search decisions and performance). As predicted, per- Overall, the current findings highlight the need of
formance success improved when students increased presenting the right answer with additional feedback
their when and what decisions (e.g., Mañá, 2011; information of how to improve further answers (i.e.,
Vidal-Abarca, Salméron, et al., 2011). Butler et al., 2013). Formative feedback just informing
In the training phase, strategic-search-decisions about the right answer was not effective enough to
feedback improved to some extent students’ strategic avoid inaccurate no-search decisions (Vidal-Abarca
search decisions over the remaining groups (e.g., use of et al., 2010) and to avoid search processes directed to
relevant information to answer questions). However, non-relevant information (Cataldo & Oakhill, 2000;
the effect of this type of feedback during the practice Cerdán & Vidal-Abarca, 2008; Cerdán et al., 2011;
process was not as clear as in the final phase regarding Vidal-Abarca et al., 2010), which led students to
all the online measures that informed about the stu- achieve poor performance in task-oriented reading.
dents’ strategic search decisions (e.g., number of Thus, these findings provide evidence about the effec-
search decisions and time searching the text per ques- tiveness of focusing the content of specific formative
tion). Accordingly, they did not improve performance feedback on how to improve students’ strategic search
over the remaining groups in the training phase. As decisions in further questions to enhance students’
exposed above, the positive impact from getting feed- search behaviours and, in turn, their performance in
back about strategic search decisions clearly appeared task-oriented reading. Moreover, the results obtained in
in the final phase when students had already practised the current study agree with TRACE model (Rouet,
with the feedback recommendation in the previous 2006) which emphasized the role of students’ strategic

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 249

search decisions to determine their performance in the Spanish Ministry of Science and Innovation with
task-oriented reading. main researcher Dr. Eduardo Vidal-Abarca. The first
In addition, in the present study, the effect of author was funded by a PhD formative grant from the
strategic-search-decisions feedback depends exclu- same institution (FPI Grant BES-2009-015677).
sively on the students’ efforts to find relevant informa-
tion to answer further questions, in contrast to previous
research that forced or induced students’ search deci- References
sions (e.g., Mañá, 2011; Vidal-Abarca, Salmerón, Ávila, V., Vidal-Abarca, E., Gil, L., & Llorens, A. C. (2010).
et al., 2011) or feedback procedures that constrained Effect of feedback on different levels of task-oriented
their strategic search decisions providing the relevant reading skills. Paper presented at 1st Joint Conference of
information to answer a question (i.e., Llorens et al., the EPS and SEPEX, Granada, Spain.
submitted). Thus, the current study contributes to show Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., &
that adaptive formative feedback delivered in a Morgan, M. (1991). The instructional effect of feedback in
computer-based system constitutes an effective instruc- test-like events. Review of Educational Research, 61, 213–
tional tool to boost self-regulation (e.g., making stra- 238.
tegic search decisions) in task-oriented reading. In fact, Bransford, J., Brown, A., & Cocking, R. (2000). How people
learn: Brain, mind, experience, and school. Washington,
one of the main advantages of using computer-based
DC: National Academy Press.
systems in education is the possibility of providing
Butler, A. C., Godbole, N., & Marsh, E. J. (2013). Explana-
immediate and adaptive formative feedback that would tion feedback is better than correct answer feedback for
aid students in correcting wrong strategies in a more promoting transfer of learning. Journal of Educational
controlled setting than human tutors can provide (i.e., Psychology, 105(2), 290–298.
Graesser, D’Mello, & Cade, 2011). Furthermore, the Butler, D. L., & Winne, P. H. (1995). Feedback and self-
current study provides evidence that supports the appli- regulated learning: A theoretical synthesis. Review of
cation of adaptive feedback about students’ strategic educational research, 65(3), 245–281.
search decisions in the context of advanced computer- Cataldo, M. G., & Oakhill, J. (2000). Why are poor
based systems, such as Intelligent Tutoring Systems, to comprehenders inefficient searchers? An investigation into
teach strategic search skills (e.g., Vidal-Abarca et al., the effects of text representation and spatial memory on the
in press). ability to locate information in text. Journal of Educational
Psychology, 92(4), 791–799.
However, the present study has some limitations.
Cerdán, R., Gilabert, R., & Vidal-Abarca, E. (2011). Select-
The feedback procedure was implemented using
ing information to answer questions: Strategic individual
non-continuous texts that include iconic and verbal differences when searching texts. Learning and Individual
information, which demands integration processes Differences, 21(2), 201–205.
(Schnotz, 2005) that might explain why students search Cerdán, R., & Vidal-Abarca, E. (2008). The effects of tasks
for more information when reading these texts than on integrating information from multiple documents.
when reading continuous texts (i.e., Serrano et al., sub- Journal of Educational Psychology, 100(1), 209–222.
mitted). Hence, further research is needed to analyse Dunlosky, J., & Lipko, A. (2007). Metacomprehension: A
the impact of strategic-search-decisions feedback in brief history and how to improve its accuracy. Current
texts that do not include iconic representations. Finally, Directions in Psychological Science, 16, 228–232.
future research would be needed to disentangle the Gick, M. L., & Holyoak, K. J. (1980). Analogical problem
effect of feedback in task-oriented reading situations solving. Cognitive Psychology, 12, 306–355.
Graesser, A. C., D’Mello, S., & Cade, W. (2011). Instruction
without the support of prompting feedback procedures
based on tutoring. In R. E. Mayer & P. A. Alexander (Eds.),
(i.e., transfer tasks).
Handbook of research on learning and instruction (pp.
408–426). New York, NY: Taylor & Francis.
Acknowledgements Graesser, A. C., Singer, M., & Trabasso, T. (1994). Con-
structing inferences during narrative text comprehension.
This research was supported by the project ‘Skills for Psychological Review, 101(3), 371.
PISA reading literacy: Processes, assessment and edu- Hattie, J., & Gan, M. (2011). Instruction based on feedback.
cational intervention’ (EDU2008-03072) granted by In R. E. Mayer (Ed.), Handbook of research on learning

© 2014 John Wiley & Sons Ltd

250 A.C. Llorens et al.

and instruction (pp. 249–271). New York: Taylor & Narciss, S. (2004). The impact of informative tutoring feed-
Francis. back and self-efficacy on motivation and achievement in
Hattie, J., & Timperley, H. (2007). The power of feedback. concept learning. Experimental Psychology, 51(3), 214–
Review of educational research, 77(1), 81–112. 228.
Kintsch, W. (1998). Comprehension: A paradigm for cogni- Organization for Economic Co-operation and Development
tion. Cambridge: Cambridge University Press. (2010). PISA 2009 results: What students know and can do
Llorens, A. C., Gil, L., Vidal-Abarca, E., Martínez, T., Mañá, – student performance in reading, mathematics and science
A., & Gilabert, R. (2011). The reading literacy test for (volume I). doi:10.1787/9789264091450-en
secondary education (CompLEC). Psicothema, 23(4), Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005).
808–817. When does feedback facilitate learning of words? Journal
Llorens, A. C., Vidal-Abarca, E., Cerdán, R., & Ávila, of Experimental Psychology. Learning, Memory, and Cog-
V. (submitted) Does formative feedback on search nition, 31, 3–8.
behavior help readers in reading literacy situations? Perfetto, G. A., Bransford, J. D., & Franks, J. J. (1983).
(Unpublished) Constraints on access in a problem solving context.
Maki, R. H. (1998). Test predictions over text material. In D. Memory and Cognition, 11, 24–31.
J. Hacker, A. C. Graesser, & J. Dunlosky (Eds.), Phye, G. D., & Sanders, C. E. (1994). Advice and feedback:
Metacognition in educational theory and practice Elements of practice for problem solving. Contemporary
(pp. 117–144). Mahwah, N.J.: Lawrence Erlbaum Educational Psychology, 19, 286–301.
Associates. Rawson, K. A., & Dunlosky, J. (2007). Improving students’
Mañá, A. (2011). Individual differences in monitoring self-evaluation of learning for key concepts in textbook
accuracy and self-regulation in task-oriented reading materials. European Journal of Cognitive Psychology,
(European PhD). University of Valencia, Valencia, 19(4–5), 559–579.
Spain. Rouet, J. F. (2006). The skills of document use: From text
Maña, A., Vidal-Abarca, E., Domínguez, C., Gil, L., & comprehension to web-based learning. Mahwah, NJ:
Cerdán, R. (2009). Papel de los procesos metacognitivos Erlbaum.
en una tarea de pregunta-respuesta con textos escritos. Rouet, J., & Vidal-Abarca, E. (2002). ‘Mining for meaning’:
Infancia y Aprendizaje, 32(4), 553–565. Cognitive effects of inserted questions in learning from
Mason, B. J., & Bruning, R. (2001). Providing feedback in scientific text. In J. Otero, J. León, & A. Graesser (Eds.),
computer-based instruction: What the research tells us The psychology of science text comprehension (pp. 417–
(Center for Instructional Innovation, University of 436). Mahwah, NJ: Lawrence Erlbaum Associates.
Nebraska–Lincoln: 14). Retrieved from http://dwb.unl Schnotz, W. (2005). An integrated model of text and picture
.edu/Edit/MB/MasonBruning.html. Accessed January comprehension. In R. E. Mayer (Ed.), Handbook of multi-
2008. media learning (pp. 49–69). Cambridge: Cambridge Uni-
McCrudden, M. T., Magliano, J., & Schraw, G. (2010). versity Press.
Exploring how relevance instructions affect personal Serrano, M., Vidal-Abarca, E., & Ferrer, A. (submitted).
reading intentions, reading goals, and text processing: A Reading strategic decisions and performance in PISA-like
mixed methods study. Contemporary Educational Psy- reading literacy tasks. (Unpublished)
chology, 35(4), 229–241. Shute, V. J. (2008). Focus on formative feedback. Review of
McCrudden, M. T., & Schraw, G. (2007). Relevance and Educational Research, 78(1), 153–189.
goal-focusing in text processing. Educational Psychology Snow, C., & RAND Reading Study Group (2002). Reading
Review, 19, 113–139. for understanding. RAND. Retrieved from http://www
McNamara, D. S., & Magliano, J. (2009). Toward a compre- .rand.org/
hensive model of comprehension. Psychology of Learning Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003).
and Motivation, 51, 297–384. Accuracy of metacognitive monitoring affects learning
Metacalfe, J., & Finn, B. (2008). Evidence that judgements of from texts. Journal of Educational Psychology, 95(1),
learning are causally related to study choice. Psychonomic 66–73.
Bulletin & Review, 15(1), 174–179. Thiede, K. W., Griffin, T. D., Wiley, J., & Redford, J. (2009).
Mory, E. H. (2004). Feedback research review. In D. Metacognitive monitoring during and after reading. In D.
Jonassen (Ed.), Handbook of research on educational com- J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook
munications and technology (pp. 745–783). Mahwah, NJ: of metacognition in education (pp. 85–106). New York,
Lawrence Erlbaum. NY: Routledge.

© 2014 John Wiley & Sons Ltd

Adaptive formative feedback 251

van den Broek, P., Young, M., Yuhtsuen, T., & Linderholm, T. Vidal-Abarca, E., Martínez, T., Salmerón, L., Cerdán, R.,
(1999). The landscape model of reading: Inferences and Gilabert, R., Gil, L., Ferris, R. (2011). Recording on-line
the online construction of memory representation. In H. processes in task-oriented reading with Read&Answer.
van Oostendorp & S. R. Goldman (Eds.), The construction Behavior Research Methods, 43, 179–192.
of mental representations during reading (pp. 71–98). Vidal-Abarca, E., Salmerón, L., & Mañá, A. (2011). Individ-
Mahwah, NJ: Lawrence Erlbaum Associates. ual differences in task-oriented reading. In M. T.
Vidal-Abarca, E., Gilabert, R., Ferrer, A., Ávila, V., McCrudden, J. P. Magliano, & G. Schraw (Eds.),
Martínez, T., Mañá, A., Llorens, A.C., Gil, L., Cerdán, R., Relevance instructions and goal-focusing in text learn-
Ramos, L., & Serrano, M. A. (In press). TuinLEC, un tutor ing (pp. 267–294). Greenwich, CT: Information Age
inteligente para mejorar la competencia lectora [TuinLEC, Publishing.
an intelligent tutoring system to improve reading literacy Whyte, M. M., Karolick, D. M., Neilsen, M. C., Elder, G. D.,
skills]. Infancia y Aprendizaje, 37(1). & Hawley, W. T. (1995). Cognitive styles and feedback in
Vidal-Abarca, E., Mañá, A., & Gil, L. (2010). Individual computer-assisted instruction. Journal of Educational
differences for self-regulating task-oriented reading activ- Computing Research, 12, 195–203.
ities. Journal of Educational Psychology, 102(4), 817–
826.

© 2014 John Wiley & Sons Ltd