Judgments of Learning

This paper reviews current known issues in student self-assessment (SSA) and identifies five topics that need further research: (1) SSA typologies, (2) accuracy, (3) role of expertise, (4) SSA and teacher/curricular expectations, and (5) effects of SSA for different students. Five SSA typologies were identified showing that there are different conceptions on the SSA components but the field still uses SSA quite uniformly. A significant amount of research has been devoted to SSA accuracy, and there is a great deal we know about it. Factors that influence accuracy and implications for teaching are examined, with consideration that students’ expertise on the task at hand might be an important prerequisite for accurate self-assessment. Additionally, the idea that SSA should also consider the students’ expectations about their learning is reflected upon. Finally, we explored how SSA works for different types of students and the challenges of helping lower performers. This paper sheds light on SSA research needs to address the known unknowns in this field.

Do we run away because we are frightened, or are we frightened because we run away? The authors address this issue with respect to the relation between metacognitive monitoring and metacognitive control. When self-regulation is goal driven, monitoring effects control processes so that increased processing effort should enhance feelings of competence and feelings of knowing. In contrast, when self-regulation is data driven, such feelings may be based themselves on the feedback from control processes, in which case they should decrease with increasing effort. Evidence for both monitoring-based control and control-based monitoring occurring even in the same situation is presented. The results are discussed with regard to the issue of the cause-and-effect relation between subjective experience and behavior.

Metamnemonic judgments probe people’s awareness of their own memory processes. The research reviewed here is an examination of the sources of information that subjects use to make judgments of learning (e.g., paired-associate judgments, ease-of-recognition predictions, free-recall judgments), and feelings of knowing (e.g., speeded strategy decisions, tip-of-the-tongue states, feeling-of-knowing judgments). The general pattern in the data suggests that subjects use different sources of information to form these judgments. Target-based sources appear to be important in judgments made at the time of acquisition, whereas cue-based judgments appear to be important in judgments made at the time of retrieval. In general, these sources of information serve as useful heuristics, and metamnemonic judgments tend to be accurate.

Three experiments investigated whether study choice was directly related to judgments of learning (JOLs) by examining people’s choices in cases in which JOLs were dissociated from recall. In Experiment 1, items were given either three repetitions or one repetition on Trial 1. Items given three repetitions received one on Trial 2, and those given one repetition received three on Trial 2—equating performance at the end of Trial 2, but yielding different immediate Trial 2 JOLs. Study choice followed the “illusory” JOLs. A delayed JOL condition in Experiment 2 did not show this JOL bias and neither did study choice. Finally, using a paradigm (Koriat & Bjork, 2005) in which similar JOLs are given to forward and backward associative pairs, despite much worse performance on the backward pairs, study choice again followed the mistaken JOLs. We concluded that JOLs—what people believe they know—directly influence people’s study choices.

We aimed to discover the neural correlates of subjective judgments of learning-whereby participants judge whether current experiences will be subsequently remembered or forgotten-and to compare these correlates to the neural correlates of actual memory formation. During event-related functional magnetic resonance imaging, participants viewed 350 scenes and predicted whether they would remember each scene in a later recognition-memory test. Activations in the medial temporal lobe were associated with actual encoding success (greater activation for objectively remembered than forgotten scenes), but not with predicted encoding success (activations did not differ for scenes predicted to be remembered versus forgotten). Conversely, activations in the ventromedial prefrontal cortex were associated with predicted but not actual encoding success, and correlated with individual differences in the accuracy of judgments of learning. Activations in the lateral and dorsomedial prefrontal cortex were associated with both actual and predicted encoding success. These findings indicate specific dissociations and associations between the neural systems that mediate actual and predicted memory formation.

We examined age differences in the heuristic used to allocate effort in learning information from sentences. Younger and older adults read and reread sentences varying in propositional density for recall, making judgments of learning before producing recall. The allocation of effort in rereading items that were less well learned on the first reading was optimized for sentences of intermediate complexity, especially for older adults. These data support a model of self-regulated learning in which readers reduce the discrepancy between current and optimal states of learning. However, self-regulation, which may be procedure based or rely on an implicit representation of the current state of learning, may be particularly efficient for older adults within a region of proximal learning.

In self-paced learning, when the regulation of study effort is goal driven (e.g., allocated to different items according to their relative importance), judgments of learning (JOLs) increase with study time. When regulation is data-driven (e.g., determined by the ease of committing the item to memory), JOLs decrease with study time (Koriat, Ma'ayan, & Nussinson, 2006). We induced learners to interpret differences in their study time (Experiment 1) or in another learner's study time (Experiment 2) as reflecting either differences in data-driven regulation or differences in goal-driven regulation. This manipulation was found to moderate the relationship of both study time and rated effort to JOLs. The results were seen to support the idea that JOLs are based on study effort but the effects of experienced effort is mediated by an attribution that intervenes between the metacognitive regulation of effort and the monitoring of one's learning. The results invite an attributional theoretical framework that encompasses both data-driven and goal-driven regulation and incorporates the option of attributing experienced effort to either or both of the two types of regulation.

The authors examined the hypothesis that judgments of learning (JOL), if governed by processing fluency during encoding, should be insensitive to the anticipated retention interval. Indeed, neither item-by-item nor aggregate JOLs exhibited “forgetting ” unless participants were asked to estimate recall rates for several different retention intervals, in which case their estimates mimicked closely actual recall rates. These results and others reported suggest that participants can access their knowledge about forgetting but only when theory-based predictions are made, and then only when the notion of forgetting is accentuated either by manipulating retention interval within individuals or by framing recall predictions in terms of forgetting rather than remembering. The authors interpret their findings in terms of the distinction between experience-based and theory-based JOLs. In recent years, social and cognitive psychologists have given increasing emphasis to a possible distinction ...

When participants studied a list of paired associates for several study-test cycles, their judgments of learning (JOLs) exhibited relatively good calibration on the 1st cycle, with a slight overconfidence. However, a shift toward marked underconfidence occurred from the 2nd cycle on. This underconfidence-with-practice (UWP) effect was very robust across several experimental manipulations, such as feedback or no feedback regarding the correctness of the answer, self-paced versus fixed-rate presentation, different incentives for correct performance, magnitude and direction of associative relationships, and conditions producing different degrees of knowing. It was also observed both in item-by-item JOLs and in aggregate JOLs. The UWP effect also occurred for list learning and for the memory of action events. Several theoretical explanations for this counterintuitive effect are discussed.

Judgments of learning (JOLs) made after a 5-min delay are almost perfectly accurate: the "delayed-JOL effect " (T. O. Nelson & J. Dunlosky, 1991). The mechanisms underlying this phenomenon have been the subject of debate. This study examined the effects of delays and short-term emory (STM) distraction on memory and metamemory (JOLs). STM distraction (2.5-30 s) immediately following encoding increased both JOL accuracy and mean cued recall. However, JOLs made after longer delays (4-5 min) were even more accurate. In addition, making a JOL at longer delays improved cued-recall performance. Conditional probabilities of cued recall (given successful initial retrieval) also increased over time and with interference, indicating that delayed JOLs may alter what they assess. Finally, increased confidence was associated with shorter JOL latencies only at delays. The results are consistent with an accessibility view of metamemory (e.g., A. Koriat, 1993). The past decade has seen a d...

Close relationship between physical space and internal knowledge representations has received ample support in the literature. For example, location of visually perceived information in vertical space has been shown to affect different numerical judgments. In addition, physical dimensions, such as weight or font size, were shown to affect judgments of learning (JOLs, an estimation of the likelihood that an item will be remembered later, or its perceived memorability). In two experiments we tested the hypothesis that differences in positioning words in vertical space may affect their perceived memorability, i.e., JOLs. In both Experiments, the words were presented in lower or in upper screen locations. In Experiment 1, JOLs were collected in the centre of the screen following word presentation. In Experiment 2, JOLs were collected at the point of word presentation and in the same location. In both experiments participants completed a free recall test. JOLs were compared between diffe...

Learners of all ages face complex decisions about how to study effectively. Here we investigated three such decisions made in concert—time allocation, ordering, and spacing. First, college students were presented with, and made judgments of learning about, 16 word-synonym pairs. Then, when presented with all 16 pairs, they created their own study schedule by choosing when and how long to

Two experiments are reported examining how value and relatedness interact to influence metacognitive monitoring and control processes. Participants studied unrelated and related word pairs, each accompanied by point values denoting how important the items were to remember. These values were presented either before or after each pair in a between-subjects design, and participants made item-by-item judgments of learning (JOLs) predicting the likelihood that each item would be remembered later. Results from Experiment 1 showed that participants used value and relatedness as cues to inform their JOLs. Interestingly, JOLs increased as a function of value even in the after condition in which value had no impact on cued recall. Participants in Experiment 2 were permitted to control study time for each item. Results showed that value and relatedness were simultaneously considered when allocating study time. These results support a cue-weighting process in which JOLs and study time allocation are based on multiple cues, which may or may not be predictive of future memory performance, and complements the agenda-based regulation model of study time (Ariel, Dunlosky, & Bailey, 2009) by providing evidence for agenda-based monitoring.

Learners of all ages face complex decisions about how to study effectively. Here we investigated three such decisions made in concert—time allocation, ordering, and spacing. First, college students were presented with, and made judgments of learning about, 16 word-synonym pairs. Then, when presented with all 16 pairs, they created their own study schedule by choosing when and how long to