The impact of intellectual ability and metacognition on learning: New support for the threshold of problematicity theory

Threshold of problematicity theory 1 1Running head: THRESHOLD OF PROBLEMATICITY THEORY This is a pre-print of the article that was published as Prins, F.J., Veenman, M.V.J., & Elshout, J.J. (2006). The impact of intellectual ability and metacognition on learning: New support for the threshold of problematicity theory. Learning & Instruction, 16 , 374-387. Copyright Elsvevier; Learning & Instruction is also available athttp://www.elsevier.com/wps/find/journaldescription.cws_home/956/description#description Frans J. Prins Educational Technology Expertise Centre, Open University of the Netherlands, Heerlen, The Netherlands Marcel V. J. Veenman Dept. of Developmental and Educational Psychology, Leiden University, The Netherlands & Graduate School of Teaching and Learning, University of Amsterdam, The Netherlands Jan J. Elshout Dept. of Psychonomics, University of Amsterdam, The Netherlands Threshold of problematicity theory 2 Abstract Three models representing different relations between intellectual ability, metacognitive skills, and learning were compared. The conditions under which each of these models holds were investigated, on the basis of the threshold of problematicity theory (Elshout, 1987). Novice and advanced learners (N = 44) passed through a computer-simulated inductive-learning environment of different complexity levels. Results show that correlational patterns between intellectual ability, metacognitive skilfulness, and learning outcomes of novice learners at the easy level were similar to the patterns of advanced learners at the intermediate level. Metacognitive skilfulness rather than intellectual ability appears essential for learning when learners operate at the boundary of their knowledge. Threshold of problematicity theory 3 The Impact of Intellectual Ability and Metacognition on Learning: New Support for the Threshold of Problematicity Theory 1. Introduction Intellectual ability and metacognition are two important determinants for learning (Veenman, 1993). Intellectual ability is regarded here as the acquired repertoire of general cognitive skills that is available to a person at a particular point of time (Humphreys, 1968; Snow & Lohman, 1984). As Anderson (1996, p. 356) phrased it, “intelligence is the simple accrual and tuning of many small units of knowledge that in total produce complex cognition. The whole is no more than the sum of its parts, but it has a lot of parts.” According to this view, performance on mental ability tests can be understood in terms of basic information-processing components (Carroll, 1993; Simon, 1976; Sternberg, 1988). This is a cognitive approach of understanding intelligence rather than the dominant psychometric approach. Whereas psychometric theories deal primarily with the structural aspects of intelligence, cognitive theories deal primarily with its processing parts (Sternberg, 1988). The concept of metacognition, first introduced by Flavell (1976), refers to both the knowledge about one’s own cognitive processes (i.e., metacognitive knowledge) and the regulation of these processes (i.e., metacognitive skills). Metacognitive knowledge concerns knowledge about the interplay between person characteristics, task characteristics, and the available strategies in a Threshold of problematicity theory 4 learning situation (Flavell, 1979), whereas metacognitive skills (i.e. executive skills; see Kluwe, 1987) concern the self-regulatory activities actually being performed by a learner in order to structure the problem solving process. The assessment of metacognitive skills through self-reports is problematic because it appears that learners have poor insight into their own behaviour (Nisbett & Wilson, 1977; Prins, Busato, Elshout, & Hamaker, 1998; Veenman, Prins, & Verheij, 2003). A valid but time-consuming method to assess metacognitive skills is the use of think-aloud protocols (Brown, 1987; Garner & Alexander, 1989; Veenman, 1993, 2005; Veenman, Elshout, & Groen, 1993). The thinkaloud method taps processes in working memory (Ericsson & Simon, 1993), which means that automatic problem solving processes cannot be measured by this method. Thus, the think-aloud method is suitable to assess metacognitive skills of novice as well as advanced learners as long as the learning task is complex enough for learners to prevent their problem-solving activities from being executed automatically. Scores for metacognitive skills measured with the think-aloud method are strongly related to metacognitive skills measured through observational data (Veenman, Kerseboom, & Imthorn, 2000) or log-file data (Veenman, Wilhelm, & Beishuizen, 2004). For years now educational researchers have been discussing the relations between intellectual ability, metacognitive skills, and learning (e.g., Davidson, Deuser, & Sternberg, 1994; Maqsud, 1997; Sternberg, 1985, 1988, 1994; Swanson, 1990; Veenman, 1993), not in the least because knowledge Threshold of problematicity theory 5 concerning these relations is essential for the design of adequate instructional support. There are reasons to consider metacognitive skills and intellectual ability as distinct concepts. First, metacognitive skills appear to be applicable over a wide range of tasks (cf. Veenman & Verheij, 2003), while mental abilities, such as verbal ability and inductive reasoning, apply to a smaller range of tasks (Sternberg, 1988; Schraw, 1998). Second, evidence implies that the frontal lobes of the brain are of critical importance for human metacognition (Metcalfe, 1996; Shimamura, 1996, 2000), whereas cognitive operations are also located in other areas of the brain (Kalat, 1992; Posner, Petersen, Fox & Raichle, 1988). Consequently, a person who has lost the most central metacognitive abilities because of brain damage “… appear to drift about like a rudderless ship”, even given a high level of cognitive abilities as measured by a variety of tests (Metcalfe, 1996, p. 404). Third, metacognitive skills are teachable and supportable (Brown & Palincsar, 1989; Schraw, 1998; Veenman, Elshout, & Busato, 1994), whereas a durable improvement of more specific cognitive abilities through training and support is rather difficult to achieve (Elshout, 1987). Considering metacognition and intellectual ability as distinct theoretical concepts does not imply that the two are unrelated. There are three models that may represent the relations between intellectual ability, metacognitive skills, and novice learning (Veenman, 1993; Veenman et al., 2004), namely the intelligence model, the independency model, and the mixed model, each of which Threshold of problematicity theory 6 are described below. Some researchers (e.g. De Corte & Van Pelt, 2003) tend to focus on which of these models is the right one. In contrast, we seek to determine the conditions under which each of these models holds. Based on Elshout’s (1987) threshold of problematicity theory, which describes the varying impact of intellectual ability on learning due to task complexity, we suggest that task complexity is a key variable here. However, the theory still has two drawbacks. First, the theory does not explicitly include the role of metacognition. Second, the empirical evidence for the threshold of problematicity theory is limited as far as realistic learning tasks are concerned. Thus, the aims of the present study are to extend the threshold of problematicity theory to the role of metacognition, and to provide empirical evidence for the theory for learners in a realistic, ill-structured, self-directed learning task. 1.1 The relation between intellectual ability and metacognitive skills The first model that represents the relations between intellectual ability, metacognitive skills, and novice learning is called the intelligence model. This model regards metacognition as a manifestation of intellectual ability. For instance, Sternberg (1985, 1988, 1994) conceived metacomponents as an essential part of human intelligence. Metacomponents are used to decide what to do, to monitor ongoing activities, and to evaluate the outcome of those activities after they have been completed and, thus, they are similar to metacognitive skills. In the same vein, the Planning, Attention, Simultaneous, and Successive Threshold of problematicity theory 7 (PASS) theory of intelligence (e.g., Das, Naglieri, & Kirby, 1994) conceives self-regulatory processes as an essential part of human intelligence. The intelligence model predicts that metacognitive skills and intellectual ability are highly correlated and that metacognitive skills will not have a predictive value for learning independent of intellectual ability. Empirical support for the intelligence model was partly found by Elshout and Veenman (1992). The second model, referred to as the independency model, predicts the opposite. Metacognitive skills and intellectual ability are not substantially correlated and, thus, they are independent determinants of learning. For instance, Allon, Gutkin, and Bruning (1994), Swanson (1990), Maqsud (1997), and Minnaert (1996) provided evidence for this model. In their studies, they found that metacognition and intellectual ability were unrelated predictors of learning. The last model, called the mixed model, predicts that metacognitive skills and intellectual ability share some variance but that metacognitive skills have a surplus value on top of intellectual ability for the prediction of leaning. Evidence for the mixed model was, for instance, obtained by Berger and Reid (1989) and in several studies by Veenman and Elshout (e.g., Veenman & Elshout, 1995, 1999; Veenman, Elshout, & Meijer, 1997). For a detailed overview concerning the evidence for the models, see Veenman, Wilhelm, and Beishuizen (2004) and Veenman and Spaans (2005). 1.2 Threshold of problematicity Threshold of problematicity theory 8 The threshold of problematicity theory (Elshout, 1987) and the mechanisms that form the base of this theory may provide insight into when and why the relations between intellectual ability, metacognition and learning outcomes change. This theory describes a varying impact of intellectual ability on learning outcomes, depending on the task complexity. At a very low and at a very high level of task complexity, the impact of intellectual ability on learning outcomes tends to be zero. Somewhere in between, at an intermediate level of task complexity, intellectual ability has maximum impact on learning outcomes. Due to the differences in domain-specific knowledge, a particular learning task is experienced as less complex by advanced learners relative to novice learners (Elshout, 1987; Snow, 1989). Therefore, the curves for novices and advanced learners will likely have different positions on the task complexity axis (see Figure 1). ---------------------------------Insert Figure 1 ---------------------------------The low or zero correlations between intellectual ability and learning outcomes at either end of the curve could be caused by a lack of variance in learning outcomes. However, in several empirical studies variance in learning outcomes at both ends of the curve was found, while intellectual ability still had little impact (Raaheim, 1988, 1991; Veenman & Elshout, 1999; Veenman & Verheij, 2003). Thus, there must be at least another determinant that causes Threshold of problematicity theory 9 the variance of learning outcomes. Elshout (1987) argued that for every person there is a critical point on the task complexity continuum, which he called the threshold of problematicity. Below this threshold, that is, for easy tasks, smooth, internalised, and fast problem-solving activities may be observed. The flow of activity is relatively automatic and algorithmic, and errors mostly come from cognitive slips rather than fundamental inadequacies of the learner. Above the threshold, that is, during more complex tasks, task-specific or domain-specific knowledge becomes increasingly inadequate. When no problem-solving strategy is available from memory, the learner must operate in a heuristic, improvisational mode (e.g., Anzai, 1991) and will shift to processes aimed at assembling a strategy. Snow (1989), for that reason, labelled the threshold of problematicity the algorithmicheuristic threshold. To be able to improvise or behave heuristically, one has to design one’s own behaviour, which is a manifestation of metacognition (Elshout, 1987). Strictly speaking, the threshold of problematicity theory just predicts the varying relation between intellectual ability and learning outcomes, and does not involve metacognition. Nevertheless, the role of metacognition can be inferred from the mechanisms described in the threshold of problematicity theory. It is hypothesised that at the right end of the curve, that is, when tasks are very complex, the quality of metacognitive skills rather than intellectual ability is the main determinant of learning outcomes, because learners have to Threshold of problematicity theory 10 improvise and use heuristics rather than call upon knowledge and skill components that are associated with intellectual ability. They need to orientate, work systematically, and evaluate their behaviour to get the initial learning process on the right track (Veenman, Prins, & Elshout, 2002). This implies a low correlation between intellectual ability and metacognitive skills. Later on during the learning process, as learners become more experienced, learning may require the cognitive sub-tasks associated with intellectual ability, which have to be regulated. Thus, the threshold of problematicity theory suggests that the mixed model holds for situations of an average complexity, that is, for a reduced but substantial zone of problematicity. Empirical evidence for the (‘pure’) threshold of problematicity theory exists. Raaheim (1988, 1991), for instance, found a part of the hypothesised curve by using a longitudinal design in which students were required to solve a particular task several times. Results showed that the impact of intellectual ability gradually increased. Veenman et al. (2002) found that in a complex computer-simulated learning environment, intellectual ability had hardly any impact on learning outcomes of novices whereas metacognitive skilfulness was the main determinant of novice learning outcomes. Finally, the general finding that intellectual ability is either unrelated or weakly related to performance of experts in several domains (Ericsson & Smith, 1991; Ericsson & Lehman, 1996) is consistent with Elshout’s (1987) theory. The empirical evidence, however, is limited and mainly focussed on the relation between Threshold of problematicity theory 11 intellectual ability and learning outcomes. In order to provide additional empirical evidence for the threshold of problematicity theory, with respect to intelligence as well as metacognitive skilfulness, a computer-simulated inductive-learning task with varying levels of complexity was developed. 1.3 Inductive learning Inductive-learning tasks require active, self-directed learning. They fit in current constructivist view of education and, probably for that reason, nowadays these task types are frequently used in secondary and higher education (De Jong & Van Joolingen, 1998). In computer-simulated inductivelearning environments, learners can design experiments by changing values of input variables and observe the resulting changes in values of output variables. By carrying out experiments, relations between input and output variables can be induced (De Jong & Van Joolingen, 1998). Thus, this kind of learning is labelled inductive learning (Holland, Holyoak, Nisbett, & Thagard, 1986). Inductive learning is a problem-solving process that can be characterized as a search process (Klahr, 2000; Klahr & Dunbar, 1988; Klahr & Simon, 1999). According to the framework of Klahr and Dunbar (1988), called scientific discovery as dual search (SDDS), learners can search in two search spaces: (1) the hypothesis space that consists of possible rules that can be induced in the learning environment, and (2) the experiment space consisting of all possible experiments that can be conducted in the learning environment. Klahr (2000) suggested that advanced learners and experts tend to search the Threshold of problematicity theory 12 hypothesis space first, and that novices tend to search the experiment space. Indeed, in complex learning environments, novice learners have limited knowledge about relevant variables and, therefore, they are particularly busy conducting experiments in order to identify independent and dependent variables before they can generate hypotheses (Van Joolingen & De Jong, 1997; Veenman et al., 2002). When task complexity increases, advanced learners may also reach a point at which their prior knowledge is insufficient and at which they have to rely on weak methods for problem solving. At that point an inductive-learning process similar to that of novices may be observed. 1.4 Research questions To obtain support for the threshold of problematicity theory, the patterns of correlations between metacognitive skilfulness, intellectual ability, and learning outcomes will be examined for novices and advanced learners for three levels of task complexity. More specifically, the unique contribution of intellectual ability and the unique contribution of metacognitive skilfulness to the variance in learning outcomes will be determined, as well as the contribution shared by intellectual ability and metacognitive skilfulness. The theory predicts that for tasks that are experienced as complex and unfamiliar, the learning process will be a heuristic, improvisational assembly process. Hence, learning outcomes for novices are probably associated with metacognitive skilfulness rather than with intellectual ability, and thus, the Threshold of problematicity theory 13 main part of the variance in learning outcomes will be accounted for by metacognitive skilfulness. The threshold of problematicity of advanced learners may be positioned at a higher level of complexity (see Figure 1). When advanced learners reach their boundary of knowledge, that is, at the most complex level of task complexity, it is expected that metacognitive skilfulness rather than intellectual ability will have substantial impact on learning outcomes. At the intermediate level, we expect a pattern in which both determinants contribute uniquely to the variance of learning outcomes. At the relative easy level of task complexity, it is hypothesized that prior knowledge will be the main determinant of learning outcomes because advanced learners may retrieve the knowledge that is necessary to complete the majority of the post-test questions directly from memory. In sum, it is expected that the pattern of correlations between metacognitive skilfulness, intellectual ability, and learning outcomes of novice learners at the easy level will be similar to the pattern of correlations of advanced learners at the most complex level. 2. Method 2.1. Participants Three months prior to this study the intellectual ability of 496 first year psychology students was assessed by a series of paper-and-pencil ability tests, representing five primary intelligence factors (Inductive reasoning, Quantitative reasoning, Verbal ability, Closure flexibility, and Sequential Threshold of problematicity theory 14 reasoning) in Carroll’s re-analyses of factor-analytic studies (Carroll, 1993). The test battery included tests for Vocabulary (difficult word meanings, 60 items), Verbal Analogies (items taking the form of e.g. foe : hatred = friend : ...?, 40 items), Conclusions (linear syllogisms, 40 items), Number Series (requiring the completion of numerical series, 30 items), Number Speed (arithmetic problems of addition, subtraction, multiplication, and division, 90 items), and Embedded Figures (discrimination of figures in complex line-patterns, 32 items). According to Sternberg (1982), verbal analogies, linear syllogisms, and number series are tests to measure reasoning skills, which are an important subset of intelligence. The unweighted mean of the z-scores on these six tests may be regarded as a measure of intellectual ability or an IQ equivalent (Veenman & Elshout, 1999). Those students, whose intellectual ability scores deviated at least 0.80 standard deviation from the mean, were classified as either being of high or relatively low intellectual ability. If participants had received physics education for three or fewer years of their six years of secondary education, they were classified as novice learners, whereas they were classified as advanced learners if they had received four years or more physics education. Thus, differences between novice and advanced learners concerned their domain knowledge, not their experiences with computer-supported learning environments. Participants in this study were 44 first-year psychology students (10 relatively low intellectual ability novices, 12 high intellectual ability novices, 13 relatively low intellectual Threshold of problematicity theory 15 ability advanced learners, and 9 high intellectual ability advanced learners). The groups did not differ in sex or age. The participants received study credits for their participation in the study. ---------------------------------Insert Figure 2 ---------------------------------2.2. Computer-simulated learning environment (the Optics Lab) In the present study a computer-simulated learning environment was used in the optics domain. In the Optics Lab learners could run experiments by manipulating light rays and lenses. Figure 2 depicts an example of an experiment in the Optics Lab. Learners could manipulate objects qualitatively by dragging them and quantitatively by changing the input of numbers. The distances between objects and the angles of light rays could be measured. Light rays were not visible during the movement of an object, so each situation after a movement of an object was considered as an observation, and therefore, as an experiment. Experiments could only be run with one lens at a time. The tasks in the Optics Lab consisted of three phases. Phase 1 dealt with refraction. In this phase participants could run experiments with four differently shaped lenses and one light source that had one light ray and could only be moved horizontally. Phase 2 dealt with focal distances of lenses. In this phase three thin lenses were available as well as one light source of three parallel light rays that could be moved horizontally and vertically. Threshold of problematicity theory 16 Phase 3 dealt with images and magnification. In this phase one light source of three divergent light rays could be moved horizontally and vertically, and the same three thin lenses of phase 2 were available. In each phase learners were asked to infer rules about optics. For instance, the assignment in phase 1 was: Try to find out, by conducting experiments, what will happen when a light ray passes through a lens. When does the emerging light ray intersect with the axis and what determines the place of this intersection point? What are the differences between the four lenses? Learners could decide how much time they would spent working on this assignment before going to the next phase and the next assignment. Each phase started with the presentation of prerequisite theory concerning that phase. In phases 2 and 3 the rules that could have been inferred in the previous phase were added to the theory, in order to reduce differences in prior knowledge associated with earlier phases. The rules that could be inferred in phase 2 were relatively easy. The relations between variables were linear, and half of the independent variables had no effect on the dependent variable. Phases 1 and 3 were more complex because of the more complex relations between system variables. These relations were non-linear and contained constraints. An example of such a relation is: If the light source is positioned very closely to a positive lens, then the emerging light ray will not intersect with the optical axis. Phase 3 was assumed to be more complex than phase 1, because in phase 3 there were Threshold of problematicity theory 17 more independent system variables and, therefore, more rules to be inferred. Although phase 1 is considered to be more complex than phase 2, learners started with phase 1 because refraction and positive and negative lenses are prerequisite concepts for understanding focal distance. Therefore, phase 1 will be labelled as the intermediate phase, phase 2 as the easy phase, and phase 3 as the complex phase of the Optics Lab. 2.3. Procedure All participants passed through the Optics Lab in a single session. The session started with a standardised twenty-minute instruction on how to operate in the Optics Lab. Participants could familiarise themselves with the Optics Lab by executing a set of prescribed actions (e.g. moving a lens in a prescribed way). After the instruction participants completed an optics pre-test, covering phases 1–3. They worked in the Optics Lab, running experiments for the next 90 minutes. Note taking was possible, using paper and pencil. Each phase of the Optics Lab ended with a post-test, which was a parallel version of the pretest. Notes could not be consulted during completion of the tests. After a time limit of one and a half hour, participants had to stop experimenting but they were allowed to finish the post-test of the Optics Lab phase in which they were working. Participants were asked to think aloud during working in Optics Lab and during completion of the post-tests. Thinking aloud protocols were taperecorded, transcribed, time tagged and analysed, in order to assess Threshold of problematicity theory 18 metacognitive skills and to examine test-taking behaviour. The computer also logged all participants’ actions in the Optics Lab in order to compare metacognitive skilfulness with actual discovery behaviour. 2.4 Metacognitive skilfulness. The quality of metacognitive skills was assessed by judging the think-aloud protocols of the participants according to the criteria of Veenman and Elshout (Veenman, 1993; Veenman & Elshout, 1991, 1995; Veenman et al., 1997). Metacognitive skilfulness was scored on four subscales: orientation activities, systematic orderliness, evaluation, and elaboration activities. Orientation activities concern the preparation for the task. These activities were judged on indications of analysing the problem statement, identifying the independent and dependent system variables, building a mental model of the task, and generating hypotheses and predictions. Judgments of systematic orderliness were based on the quality of planning activities, the systematic execution of those plans, completing an orderly sequence of actions, and the avoidance of unsystematic events (such as varying two variables at the same time). Evaluation activities concern the control of the learning process. They were judged on monitoring and checking, both on the local level (e.g. detecting errors and checking calculations) as well as on the global level of keeping track of progress being made (e.g. verifying whether the results obtained provide an answer to the problem statement). Finally, judgments of elaboration concern the intention of storing of findings and concepts in Threshold of problematicity theory 19 memory. They were based on indications of recapitulating, drawing conclusions, relating these conclusions to the subject matter, and generating explanations. Elaboration itself may be conceived as a cognitive activity, but it is assumed that the occurrence of such cognitive activity at an appropriate point in time results from metacognitive activity. In order to avoid contamination of protocol scores with learning outcomes, aspects of metacognitive skilfulness were judged on the quality of performing regulatory activities, not on the correctness of the information that resulted from these activities. For instance, generating well-considered, though incorrect predictions or incorrect conclusions may still result in high scores on orientation or elaboration. The four subscales of metacognitive skilfulness were rated on a fivepoint scale, ranging from 0 to 4. For each participant summed scores over the four subscales were computed separately for the easy, the intermediate, and the complex phase, thus obtaining three scores for metacognitive skilfulness. The first author, who received no prior information about the participant’s scores of intellectual ability, judged the think-aloud protocols. In an earlier study (Veenman et al., 2002), the first two authors judged similar protocols obtained in Optics Lab according to the same criteria. Average Cronbach’s alphas over the four different subscales of .93 were established as inter-judge reliabilities. 2.5 Learning outcomes. Threshold of problematicity theory 20 Pre-tests and post-tests consisted of three types of questions: (1) qualitative WHAT-IF questions, (2) qualitative reasoning questions, and (3) quantitative questions. WHAT-IF questions contained three parts: conditions, action, and predictions (Swaak & De Jong, 1996). In all questions conditions were defined by a depicted situation with a light source, light rays and a lens. Actions and predictions were presented in text. Actions referred to a change in the value of an independent variable, while predictions referred to the changed value of a dependent variable. The learner was asked to decide which of the three predicted states would match the conditions and the action. An example of a WHAT-IF question: The light source is moved a bit to the right. Where will the emerging light ray intersect? (1) Closer to the lens, (2) At the same distance from the lens, or (3) Further away from the lens. Qualitative reasoning questions were reversed WHAT-IF questions: A prediction was presented and learners had to choose which of the three actions would match the conditions and the prediction. An example of a qualitative reasoning question: When will the emerging light rays intersect above the optical axis? (1) If you rotate the light rays upwards, (2) If you move the light source a bit downwards, (3) If you move the lens a bit to the right. Quantitative questions also contained depicted situations, but in this case the answer categories concerned numbers. In the easy phase the quantitative questions of the post-test dealt with the focal distance of the available lenses, which made them inappropriate for inclusion in a pre-test. In the complex phase the quantitative questions concerned magnification and Threshold of problematicity theory 21 the quantitative relation between object distance and image distance. The intermediate phase contained no quantitative questions because with a light source that has one light ray it is impossible to question magnification in a quantitative way. All independent variables in the Optics Lab were systematically questioned with the WHAT-IF items, the qualitative reasoning questions and the quantitative questions. The pre-test contained, in random order, 14 questions concerning the easy phase (10 WHAT-IF and 4 qualitative reasoning), 15 questions concerning the intermediate phase (10 WHAT-IF and 5 qualitative reasoning), and 28 questions concerning the complex phase (18 WHAT-IF, 3 qualitative reasoning, and 7 quantitative questions). The post-test questions were parallel versions of the pre-test questions, supplemented with 3 quantitative questions concerning the easy phase. In the results section, the qualitative questions (WHAT-IF and the qualitative reasoning) and the quantitative questions will be reported separately. 3. Results 3.1 Analyses prerequisite Cronbach’s alphas of the qualitative pre-test of the easy, the intermediate, the complex phase and the total test were .65 (14 items), .04 (15 items), .30 (21 items) and .55 (50 items), respectively, whereas Cronbach’s alphas of the qualitative post-test of the easy, the intermediate, the complex phase, and the total test were .48 (14 items), .59 (15 items), .53 (21 items) and .62 (50 items), Threshold of problematicity theory 22 respectively. Unfortunately, reliability coefficients of the quantitative tests were insufficient. Cronbach’s alphas of the quantitative pre-test of the complex phase, the post-test of the easy phase, and the complex phase were . 01 (7 items), .24 (3 items), and .21 (7 items), respectively. Thus, results concerning the quantitative tests were excluded from further analyses. Table 1 depicts the means and standard deviations of the Optics Lab pretest scores for relatively low and high intellectual ability novice and advanced learners. Pre-test scores are presented for each phase separately and for the total test. A MANOVA revealed main effects for level of expertise. Advanced learners outperformed novices on the Optics Lab qualitative pre-test of the easy phase, F(1,40) = 5.44, p < .05, the complex phase, F(1,40) = 11.75, p < .01, and the total pre-test, F(1,40) = 15.63, p < .01. No main effect for intellectual ability, F(3, 38) = 1.24, p = .31, and no interaction effect of level of expertise with intellectual ability for pre-test scores, F(4, 37) = 0.59, p = .68, were found. Table 2 depicts the means and standard deviations of the Optics Lab post-test scores for each phase separately and for the total test for relatively low and high intellectual ability novice and advanced learners. One-tailed ttests showed that novice learners gained qualitative knowledge from pre- to post-test in the easy phase, t(22) = 6.05, p < .01, the intermediate phase, t(22) = 4.31, p < .01, and in the complex phase, t(21) = 2.73, p < .01. Consequently, their total qualitative post-test score was significantly higher than their pre-test score, t(21) = 6.31, p < .01. The advanced learners gained qualitative Threshold of problematicity theory 23 knowledge in the easy phase, t(20) = 4.90, p < .01], the intermediate phase, t(20) = 2.35, p < .05, and the complex phase, t(18) = 1.77, p < .05, as well. Their total qualitative post-test score was significantly higher than their pre-test score, t(18) = 5.02, p < .01. As expected, the pre-test and post-test scores of the easy phase correlated significantly for advanced learners (r = .54, p < .05, n = 21), but not for novice learners (r = .11, p = .61, n = 23), indicating that in the easy phase prior knowledge is an important determinant for learning for advanced learners. The pre-test and post-test scores of the intermediate phase correlated significantly for novice learners (r = .49, p < .05) but not for advanced learners (r = .21, p = .37). Other correlations between pre-test and post-test scores were not significant. A MANOVA with repeated measures indicated that the knowledge gain on the total qualitative test was significantly higher for novice learners than for advanced learners, F(1, 39) = 4.43, p < .05. The two groups differed on the pre-test, but not on the post-test. Furthermore, this MANOVA showed that the knowledge gain for high intellectual ability learners in the intermediate phase was larger than the knowledge gain for relatively low intellectual ability learners (Mlowpre = 5.36, SDlowpre = 1.76; Mhighpre = 5.82, SDhighpre = 1.89; Mlowpost = 6.36, SDlowpost = 2.08; Mhighpost = 8.55, SDhighpost = 2.82), F(1, 42) = 5.18, p < .05. Threshold of problematicity theory 24 3.2 Metacognitive skilfulness and the relation with intellectual ability A measure for metacognitive skilfulness, consisting of a summed score of orientation activities, systematic orderliness, evaluation, and elaboration activities, was assessed for each phase separately. The correlations between measures of metacognitive skilfulness of the easy phase and the intermediate phase (phase 2 and 1), the intermediate phase and the complex phase (phase 1 and 3), and the easy phase and the complex phase (phase 2 and 3) were respectively .40 (p < .01), .19 (p = .12), and .54 (p < .01). A principal component analysis on the measures of metacognitive skilfulness for the three phases of the Optics Lab extracted one component with an eigenvalue greater than 1.0. This component had an eigenvalue of 1.76, with 58.8% of the variance accounted for. The eigenvalue of the second component was 0.83, representing an additional 27.5% of the variance. In Table 3 the unrotated component matrix is depicted. All scores loaded high and positive on the first component. Therefore, the results concerning the overall score for metacognitive skilfulness, that is, the summed score over the measures for metacognitive skilfulness of the three phases of the Optics Lab, will be reported below. The results concerning the overall score for metacognitive skilfulness were very similar to the results concerning the measures of metacognitive skilfulness of the three phase of the Optics Lab. Threshold of problematicity theory 25 Novice and advanced learners did not differ on overall metacognitive skilfulness (Msumnov = 23.77, SDnov = 9.73; Msumadv = 19.89, SDadv = 10.03), t(39) = 1.26, p = .22. Furthermore, no difference between novice and advanced learners was found for intellectual ability scores (Mnov = 0.03, SDnov = 0.95; Madv = – 0.06, SDadv = 1.00, respectively). The correlations between overall metacognitive skilfulness and intellectual ability were .54 (p < .01) for novice learners, and .60 (p < .01) for advanced learners. After correction for selection of extreme groups on intellectual ability (Gulliksen, 1961), these correlations became .39 and .44, respectively. 3.3 Determinants of learning Table 4 shows the correlations between intellectual ability and learning outcomes, and the correlations between overall metacognitive skilfulness and learning outcomes. Correlations are presented for novice and advanced learners separately for qualitative post-tests of each phase and for the total tests. The uncorrected correlations as well as corrected correlations for selection of extreme groups on intellectual ability (Gulliksen, 1961) are shown. Inspection of the bivariate plots did not reveal a violation of the requirement of homoskedasticity. It should be pointed out that correlations for indirect selection (here the corrected correlation between metacognitive skilfulness and learning outcomes, see last column of Table 4) lack a statistical method for establishing a significance level. The significance of correlations for direct Threshold of problematicity theory 26 selection (correlations with intellectual ability) is identical to the significance of the uncorrected correlations (Elshout, Overbeek, Roe, & Vijn, 1979). Table 5 depicts the distributions of the variance in learning outcomes of the three phases of the Optics Lab. Semipartial correlations (Nunnally, 1967) were calculated by partialing intellectual ability from the correlations between metacognitive skilfulness and learning outcomes, and vice versa. They represent the unique contribution of intellectual ability and the unique contribution of metacognitive skilfulness to the variance of learning outcomes. The correlations in table 4 and the distributions in Table 5 show that metacognitive skilfulness is the main determinant of learning outcomes in the easy phase for novices, as expected. Moreover, for advanced learners, metacognitive skilfulness is the main determinant of learning outcomes in the intermediate phase. In the other phases the unique contributions of both intellectual ability and metacognitive skilfulness to the variance in learning outcomes is modest for both novice and advanced learners. 4. Discussion The aims of the present study were to extend the threshold of problematicity theory (Elshout, 1987) to the role of metacognition, and to provide empirical evidence for this theory for learners in a realistic, ill-structured, self-directed learning task. We examined whether the task complexity would cause changes in the patterns of correlation between intellectual ability, metacognitive skilfulness, and learning outcomes, as the threshold of Threshold of problematicity theory 27 problematicity theory suggests. It was hypothesized that the pattern of correlations between intellectual, ability, metacognitive skilfulness, and learning outcomes would differ for novice and advanced learners. The pattern for novice learners in the relatively easy phase of the Optics Lab was expected to be similar to the pattern for advanced learners in the complex phase. As was expected, the pattern of correlations for novice and advanced learners differed substantially. For novice learners, metacognitive skilfulness was the main determinant for learning outcomes in the relative easy phase of the Optics Lab, which is shown by the size of unique contribution of metacognitive skilfulness to the post-test scores of the easy phase and the small unique contribution of intellectual ability. In the intermediate and complex phase, the impact of metacognitive skilfulness and intellectual ability on learning outcomes was limited. Considering the relatively low impact of intellectual ability on learning outcomes for novice learners, even the easy phase of the Optics Lab may have been be positioned beyond their threshold of problematicity. For advanced learners, in contrast, metacognitive skilfulness was the main determinant in the intermediate phase, and not, as we expected, in the complex phase. It is likely that, because of the complexity of the Optics Lab (see also Veenman et al., 2002), the advanced learners already reached their boundary of knowledge in the intermediate phase. Thus, the pattern of Threshold of problematicity theory 28 correlations found for advanced learners in the intermediate phase resembled the pattern of correlations found for novice learners in the easy phase. These findings about the varying impact of metacognitive skills are in line with ideas of Weinert (1987), who stated that only for tasks of medium difficulty, where strategic solutions are possible, learning behaviour and performance could be positively influenced by metacognitive skills. On extremely difficult tasks, the use of metacognition may lead to the realisation that further effort will not be productive. In spite of the rather high complexity of the Optics Lab, novice learners were indeed able to acquire knowledge in the Optics Lab, which suggests that the task itself was not extremely difficult. In fact, the knowledge gain of novice learners was larger than the knowledge gain of the advanced learners, without the risk of a ceiling effect for advanced learners on the pre-test. The knowledge gains of novice learners were even large enough to let initial differences on the pretests disappear on the post-tests. Furthermore, as expected, for advanced learners prior knowledge was the main determinant for learning outcomes in the easy phase, shown by a high positive correlation between pre-test and post-test scores in that phase and the modest correlations between intellectual ability and metacognitive skilfulness at the one hand, and post-test scores on the other. Apparently, advanced learners could retrieve the knowledge that is necessary to complete the post-test questions directly from memory. In the complex phase, a Threshold of problematicity theory 29 marginal significant correlation between intellectual ability and learning outcomes was found for advanced learners. These results are in accordance with the findings of Veenman and Elshout (1999). In sum, the varying impact of intellectual ability on learning outcomes found in the present study is in accordance with the threshold theory, which states that the threshold of problematicity of advanced learners is positioned at a higher level of the objective task complexity than the threshold of problematicity of novice learners. Also the role of metacognition is in accordance with our theoretical view. In general, the patterns of correlations found in the present study resemble the mixed model (Veenman, 1993): metacognitive skilfulness and intellectual ability were related to some extent, while they both had an independent impact on qualitative learning outcomes at an adequate level of task complexity. Reliability scores of the qualitative test were sufficient, except for some reliability scores of the qualitative pre-tests. This could cause problems interpreting the correlations concerning these scores. Probably, low reliability was partly due to the complexity of the test items. When learners have limited knowledge, as is the case with novice learners, or when misconceptions are developed, inter-item correlations and, therefore, reliability will decrease. It is well known that the internal consistency of novices in a pre-test is often low. Also the rather limited size of the tests and the fact that questions were multiple-choice items decreased test reliability. However, it should be noted Threshold of problematicity theory 30 that Cronbach’s alpha is a lower bound measure for reliability and, thus, likely underestimates reliability. For this reason, correlations concerning scales with moderate reliability could be higher than one could expect on account of Cronbach’s alpha. Moreover, low internal consistency is a big problem when results are not significant, because low internal consistency would be the alternative explanation for a lack of significant results. When significant results are found, moderately low internal consistency is not a pressing issue. Reliability scores of the quantitative tests, however, were too low, probably for the above-mentioned reasons, and results concerning the quantitative tests had to be excluded from further analyses. When educators design learning environments and learning tasks, the threshold of problematicity of the students should be taken into account. Operating slightly above one’s threshold is the better way to learn (Elshout, 1987; Snow, 1989) because learners are then challenged to extend their heuristic and improvisational abilities. When learners operate too far beyond their threshold, the heuristic behaviour will be ineffective. Problem solving as well as learning will then become almost an impossible venture (see Weinert, 1987). On the other hand, learners working far below their thresholds may only strengthen habitual automatic performance. Indeed, successful assembly processes will be strengthened and automatised by continued practice. They become, in Veenman’s (1993) terms, domain-specific task schemata, which are stored in memory, independently of the general task schema from which they Threshold of problematicity theory 31 were derived. Instructional support, however, may be necessary to adapt the task complexity to the threshold of the learner. Eventually, continued practice will gradually raise a learner’s threshold of problematicity for a particular task: More and more difficult instances of a task type become nonproblematical and automatic (Elshout, 1987). It will be a challenge for further research to try to move the threshold of problematicity of novice learners upwards by providing specific instructional support aimed at supporting their metacognitive skilfulness. Good examples of this kind of support are structuring the environment by providing subassignments and so-called telling experiments (Veenman & Elshout, 1995), in which learners are explicitly told which experiments they have to conduct, leading to better learning performances. This kind of adaptive support may allow both high and low intelligent novice learners to turn more rapidly from generating hypotheses to data interpretation and testing hypotheses. Threshold of problematicity theory 32 References Allon, M., Gutkin, T. B., & Bruning, R. (1994). The relationship between metacognition and intelligence in normal adolescence: Some tentative but surprising findings. Psychology in the Schools, 31, 93-97. Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American Psychologist, 51, 355-365. Anzai, Y. (1991). Learning and use of representations for physics expertise. In K. A. Ericsson & J. Smith (Eds.), Towards a general theory of expertise (pp. 6492). Cambridge, MA: Cambridge University Press. Berger, R. S., & Reid, D. K. (1989). Differences that make a difference: Comparisons of metacomponential functioning and knowledge base among groups of high and low IQ learning disabled, mildly mentally retarded, and normally achieving adults. Journal of Learning Disabilities, 22, 422-429. Brown, A. L. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 65-116). Hillsdale, NJ: Lawrence Erlbaum Associates. Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in the honor of Robert Glaser (pp. 393-451). Hillsdale, NJ: Lawrence Erlbaum Associates. Threshold of problematicity theory 33 Carroll, J. B. (1993). Human cognitive abilities. A survey of factor-analytic studies. Cambridge: Cambridge University Press. Corte, E. De, & Pelt, E. Van (2003, May). De relatie tussen metacognitieve vaardigheden en intelligentie in de hogere leerjaren van de lagere school: Een exploratief onderzoek [The relation between metacognitive skills and intellectual ability in the higher learning years of primary school: An explorative study]. Paper presented at the annual meeting of the Netherlands Educational Research Association (NERA), the Educational Research Days, Kerkrade, the Netherlands. Das, J. P., Naglieri, J. A., & Kirby, J. R. (1994). Assessment of cognitive processes. Boston: Allyn and Bacon. Davidson, J. E., Deuser, R., & Sternberg, R. J. (1994). The role of metacognition in problem solving. In J. Metcalfe & P. Shimamura (Eds.) Metacognition, knowing about knowing (pp. 207-226). Cambridge: MIT Press. Elshout, J. J. (1987). Problem solving and education. In E. de Corte, H. Lodewijks, R. Parmentier, & P. Span (Eds.), Learning and Instruction (pp. 259-273). Oxford: Pergamon Books Ltd. Leuven: University Press. Elshout, J. J., Overbeek, H. van, Roe, R. A., & Vijn, P. (1979). Testing the hypothesis that RHO = ) in selected samples (case 1). Educational and Psychological Measurement, 39, 573-576. Threshold of problematicity theory 34 Elshout, J. J., & Veenman, M. V. J. (1992). Relation between intellectual ability and working method as predictors of learning. Journal of Educational Research, 85, 134-143. Ericsson, K. A., & Lehman, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual Review of Psychology, 47, 273-305. Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis: Verbal reports as data. Cambridge, MA: MIT Press. Ericsson, K. A., & Smith, J. (1991). Prospects and limits of the empirical study of expertise: an introduction. In K. A. Ericsson & J. Smith (Eds.), Towards a general theory of expertise (pp. 1-38). Cambridge: Cambridge University Press. Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231-235). Hillsdale, NJ: Lawrence Erlbaum Associates. Flavell, J. H. (1979). Metacognition and cognitive monitoring. A new area of cognitive developmental inquiry. American Psychologist, 34, 906-911. Garner, R., & Alexander, P. A. (1989). Metacognition: answered and unanswered questions. Educational Psychologist, 24, 143-158. Gulliksen, H. (1961). Theory of mental tests. New York: Wiley. Holland, J. H., Holyoak, K. J., Nisbett, R. E., & Thagard, P. R. (1986). Induction: Processes of inference, learning, and discovery. Cambridge, MA: MIT Press. Threshold of problematicity theory 35 Humphreys, L. G. (1968). The fleeting nature of the prediction of college academic success. Journal of Educational Psychology, 59, 375-380. Jong, T. de, & Joolingen, W. R. van (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68, 179-201. Joolingen, W. R. van, & Jong, T. de (1997). An extended dual search space model of scientific discovery learning. Instructional Science, 25, 307-346. Kalat, J. W. (1992). Biological Psychology. Belmont, CA: Wadsworth Publishing Company. Klahr, D. (2000). Exploring science: The cognition and development of discovery processes. Cambridge, MA: MIT Press. Klahr, D., & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12, 1-48. Klahr, D., & Simon, H. A. (1999). Studies of scientific discovery: complementary approaches and convergent findings. Psychological Bulletin, 125, 524-543. Kluwe, R. H. (1987). Executive decisions and regulation of problem solving behavior. In F. E. Weinert, & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 31-64). Hillsdale, N.J.: Erlbaum. Maqsud, M. (1997). Effects of metacognitive skills and nonverbal ability on academic achievement of high school pupils. Educational Psychology, 17, 387-397. Threshold of problematicity theory 36 Metcalfe, J. (1996). Metacognitive processes. In E. L. Bjork & R. A. Bjork (Eds.), Memory (pp. 381-407). San Diego, CA: Academic Press. Minnaert, A. (1996). Can metacognition compensate for intelligence in the first year of Belgian higher education? Psychologica Belgica, 36, 227-244. Nisbett, R. E. & Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychological Review, 84, 231-259. Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill. Posner, M. I., Petersen, S. E., Fox, P. T., & Raichle, M. E. (1988). Localization of cognitive operations in the human brain. Science, 240, 1627-1631. Prins, F. J., Busato, V. V., Elshout, J. J., & Hamaker, C. (1998). Een nieuwe bijdrage tot de validatie van het (meta)cognitieve deel van de Inventaris Leerstijlen [A new contribution to the validation of the cognitive and metacognitive part of the Inventory of Learning Styles]. Pedagogische Studien, 75, 73-93. Raaheim, K. (1988). Intelligence and task novelty. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (pp. 73-97). Hillsdale, NJ: Lawrence Erlbaum Associates. Raaheim, K. (1991). Is the high IQ person really in trouble? Why? In H. A. H. Rowe (Ed.), Intelligence: Reconceptualization and measurement (pp. 35-46). Hillsdale, NJ: Lawrence Erlbaum Associates. Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125. Threshold of problematicity theory 37 Simon, H. A. (1976). Identifying basic abilities underlying intelligent performance of complex tasks. In L. B. Resnick (Ed.), The nature of intelligence (pp. 65-98). Hillsdale, NJ: Lawrence Erlbaum Associates. Shimamura, A. P. (1996). The role of the prefrontal cortex in controlling and monitoring memory processes. In L. M. Reder (Ed.), Implicit memory and metacognition (pp. 259-274). Mahwah, NJ: Lawrence Erlbaum Associates. Shimamura, A. P. (2000). What is metacognition? The brain knows [Review of the book Metacognition and cognitive neuropsychology: Monitoring and control processes]. American Journal of Psychology, 113, 142-146. Snow, R. E. (1989). Aptitude-treatment interaction as a framework for research on individual differences in learning. In P. H. Ackerman, R. J. Sternberg, & R. Glaser (Eds.), Learning and individual differences: Advances in theory and research (pp.13-59). New York: Freeman. Snow, R. E., & Lohman, D. F. (1984). Toward a theory of cognitive aptitude for learning from instruction. Journal of Educational Psychology, 76, 347-376. Sternberg, R. J. (1982). Reasoning, problem solving, and intelligence. In R. J. sternberg (Ed.), Handbook of human intelligence (pp. 225-307). Cambridge; Cambridge University Press. Sternberg, R. J. (1985). Beyond IQ: A triarchic theory of human intelligence. Cambridge: Cambridge University press. Threshold of problematicity theory 38 Sternberg, R. J. (1988). Intelligence. In R. J. Sternberg & E. E. Smith (Eds.), The psychology of human thought (pp. 267-308). Cambridge: Cambridge University Press. Sternberg, R. J. (1994). Intelligence. In R. J. Sternberg (Ed.), Thinking and problem solving (pp. 179-213). San Diego, CA: Academic Press. Swaak, J., & Jong, T. de (1996). Measuring intuitive knowledge in science: the development of the what-if test. Studies in Educational Evaluation, 22, 341362. Swanson, H. L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82, 306-314. Veenman, M. V. J. (1993). Intellectual ability and metacognitive skill: Determinants of discovery learning in computerized learning environment. Doctoral dissertation, University of Amsterdam, The Netherlands. Veenman, M. V. J. (2005). The assessment of metacognitive skills: What can be learned from multi-method designs? In C. Artelt, & B. Moschner (Eds), Lernstrategien und Metakognition: Implikationen für Forschung und Praxis. Berlin: Waxmann. Veenman, M. V. J., & Elshout, J. J. (1991). Intellectual ability and working method as predictors of novice learning. Learning and Instruction, 1, 303-317. Veenman, M. V. J., & Elshout, J. J. (1995). Differential effects of instructional support on learning in simulation environments. Instructional Science, 22, 363-383. Threshold of problematicity theory 39 Veenman, M. V. J., & Elshout, J. J. (1999). Changes in the relation between cognitive and metacognitive skills during the acquisition of expertise. European Journal of Psychology of Education, 14, 509-523. Veenman, M. V. J., Elshout, J. J., & Busato, V. V. (1994). Metacognitive mediation in learning with computer-based simulations. Computers in Human Behavior, 10, 93-106. Veenman, M. V. J., Elshout, J. J., & Groen, M. G. M. (1993). Thinking aloud: does it affect regulatory processes in learning? Tijdschrift voor Onderwijsresearch, 18, 322-330. Veenman, M. V. J., Elshout, J. J., & Meijer, J. (1997). The generality vs. domain specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7, 187-209. Veenman, M. V. J., Kerseboom, L., & Imthorn, C. (2000). Test anxiety and metacognitive skillfulness: Availablity versus production deficiencies. Anxiety, Stress, and Coping, 13, 391-412. Veenman, M. V. J., Prins, F. J., & Elshout, J. J. (2002). Initial inductive learning in a complex computer simulated environment: the role of metacognitive skills and intellectual ability. Computers in Human Behavior, 18, 327-342. Veenman, M. V. J., Prins, F. J., & Verheij, J. (2003). Learning styles: Self-reports versus thinking-aloud measures. British Journal of Educational Psychology, 73, 357-372. Threshold of problematicity theory 40 Veenman, M. V. J., & Spaans, M. (2005). Relation between intellectual ability and metacognitive skills: Age and task differences. Learning and Individual Differences, 15, 159-176. Veenman, M. V. J. & Verheij, J. (2003). Identifying technical students at risk: Relating general versus specific metacognitive skills to study success. Learning and Individual Differences, 13, 259-272. Veenman, M. V. J., Wilhelm, P. & Beishuizen, J. J. (2004). The relation between intellectual and metacognitive skills from a developmental perspective. Learning and Instruction, 14, 89-109. Weinert, F. E. (1987). Introduction and overview: Metacognition and motivation as determinants of effective learning and understanding. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 1-16). Hillsdale, NJ: Lawrence Erlbaum Associates. Threshold of problematicity theory 41 Acknowledgements This research was financially supported by the Foundation for Behavioural Sciences (SGW), part of the Netherlands Organisation for Scientific Research, with grant 575-22-003. This research project was hosted by the Department of Developmental and Educational Psychology at Leiden University from 1996 to 2000, to which department the first author was affiliated to during this period. The authors thank the colleagues of the Research Department of the Educational Technology Expertise Center, especially Jan-Willem Strijbos, for their valuable comments on an earlier draft. Threshold of problematicity theory 42 Table 1 Means and standard deviations of the qualitative pre-test scores in the different phases of the Optics Lab for novice and advanced learners with relatively low and high intellectual ability (IA) PRE-TEST Low IA M High IA SD n M SD Total n M SD n Novice Easy phase 6.50 2.01 10 7.23 2.62 13 6.91 2.35 23 Intermediate phase 5.00 1.76 10 5.46 2.22 13 5.26 2.00 23 Complex phase 6.80 1.55 10 8.85 2.34 13 7.96 2.25 23 18.30 3.40 10 21.54 3.84 13 20.13 3.93 23 Total Advanced Easy phase 8.92 3.03 12 8.67 3.04 9 8.81 2.96 21 Intermediate phase 5.67 1.78 12 6.33 1.22 9 5.95 1.56 21 Complex phase 9.92 2.15 12 10.33 2.65 9 10.10 2.32 21 24.50 4.98 12 25.33 4.09 9 24.86 4.53 21 Total Threshold of problematicity theory 43 Table 2 Means and standard deviations of the qualitative post-test scores in the different phases of the Optics Lab for novice and advanced learners with relatively low and high intellectual ability (IA) POST-TEST Low IA M High IA SD n M Total SD n M SD n Novice Easy phase 9.80 1.62 10 11.38 2.29 13 10.70 2.14 23 Intermediate phase 6.40 2.07 10 8.46 3.15 13 7.57 2.87 23 Complex phase 10.44 3.32 9 10.92 4.03 13 10.73 3.68 22 Total 26.89 3.98 9 30.77 6.66 13 29.18 5.93 22 Advanced Easy phase 11.63 1.30 12 11.22 1.48 9 11.48 1.36 21 6.33 2.19 12 8.67 2.45 9 7.33 2.54 21 Complex phase 10.20 1.48 10 11.89 2.37 9 11.00 2.08 19 Total 28.50 3.44 10 31.78 3.23 9 30.05 3.66 19 Intermediate phase Threshold of problematicity theory 44 Table 3 Unrotated component matrix for measures of metacognitive skilfulness for three different phases of the Optics Lab Easy phase Intermediate phase Complex phase Component 1 .87 .64 .77 Component 2 –.09 .74 –.51 Threshold of problematicity theory 45 Table 4 Correlations between intellectual ability (IA), overall metacognitive skilfulness (MS) and post-test scores in the different phases of the Optics Lab for novice and advanced learners Uncorrected IA Corrected MS IA MS Novice (n = 22) Easy phase .36 .49* .24 .43 Intermediate phase .36 .37 .25 .29 Complex phase .08 .15 .05 .13 Total .35 .45* .24 .38 –.11 .19 Advanced (n = 19) Easy phase –.17 .11 Intermediate phase .49* .72** .34* .67 Complex phase .45* .26 .31* .13 Total .52* .68** .37* .61 Note. Corrected = corrected for selection of extreme groups of intellectual ability. * p < .05. ** p < .01. Threshold of problematicity theory 46 Table 5 Proportion of the variance of post-test scores in the different phases of the Optics Lab accounted for by intellectual ability (IA) and metacognitive skilfulness (MS) for novice and advanced learners IA unique MS unique IA and MS Total Novice (n = 22) Easy phase .01 .13 .05 .19 Intermediate phase .02 .04 .04 .11 Complex phase .00 .01 .00 .02 Total .01 .10 .05 .15 Advanced (n = 19) Easy phase .05 .07 .03 .15 Intermediate phase .00 .34 .11 .45 Complex phase .08 .00 .02 .10 Total .01 .25 .12 .38 Threshold of problematicity theory 47 Impact of intellectual ability on learning Advanced learners Novice learners Increasing task complexity Figure 1. The relation between task complexity and the impact of intellectual ability on learning outcomes. Threshold of problematicity theory 48 Figure 2. An example of an experiment in the Optics Lab.