2014 IEEE 14th International Conference on Advanced Learning Technologies
A meta-analysis on the effectiveness of Intelligent Tutoring Systems (ITS) in computer science ed... more A meta-analysis on the effectiveness of Intelligent Tutoring Systems (ITS) in computer science education compared the learning outcomes of ITS and non-ITS instruction. A search of the literature found 22 effect sizes (involving 1,447 participants) that met the pre-defined inclusion criteria. Although most of the ITS were used to teach programming, other topics such as database design and computer literacy were also represented. There was a significant overall effect size favoring the use of ITS. There was a significant advantage of ITS over teacher-led classroom instruction and non-ITS computer-based instruction. ITS were more effective than the instructional methods to which they were compared regardless of whether they modeled misconceptions and regardless of whether they were the primary means of instruction or were an integrated component of learning activities that included other means of instruction.
Intelligent Tutoring Systems (ITS) are computer programs that model learners’ psychological state... more Intelligent Tutoring Systems (ITS) are computer programs that model learners’ psychological states to provide individualized instruction. They have been developed for diverse subject areas (e.g., algebra, medicine, law, reading) to help learners acquire domain-specific, cognitive and metacognitive knowledge. A meta-analysis was conducted on research that compared the outcomes from students learning from ITS to those learning from non-ITS learning environments. The meta-analysis examined how effect sizes varied with type of ITS, type of comparison treatment received by learners, type of learning outcome, whether knowledge to be learned was procedural or declarative, and other factors. After a search of major bibliographic databases, 107 effect sizes involving 14,321 participants were extracted and analyzed. The use of ITS was associated with greater achievement in comparison with teacher-led, large-group instruction (g = .42), non-ITS computer-based instruction (g = .57), and textbooks or workbooks (g = .35). There was no significant difference between learning from ITS and learning from individualized human tutoring (g = –.11) or small-group instruction (g = .05). Significant, positive mean effect sizes were found regardless of whether the ITS was used as the principal means of instruction, a supplement to teacher-led instruction, an integral component of teacher-led instruction, or an aid to homework. Significant, positive effect sizes were found at all levels of education, in almost all subject domains evaluated, and whether or not the ITS provided feedback or modeled student misconceptions. The claim that ITS are relatively effective tools for learning is consistent with our analysis of potential publication bias.
Proceedings of the 14th European Workshop on Natural Language Generation, 2013
When instructors prepare learning materials for students, they frequently develop accompanying qu... more When instructors prepare learning materials for students, they frequently develop accompanying questions to guide learning. Natural language processing technology can be used to automatically generate such questions but techniques used have not fully leveraged semantic information contained in the learning materials or the full context in which the question generation task occurs. We introduce a sophisticated template-based approach that incorporates semantic role labels into a system that automatically generates natural language questions to support online learning. While we have not yet incorporated the full learning context into our approach, our preliminary evaluation and evaluation methodology indicate our approach is a promising one for supporting learning.
An animated concept map represents verbal information in a node-link diagram that changes over ti... more An animated concept map represents verbal information in a node-link diagram that changes over time. The goals of the experiment were to evaluate the instructional effects of presenting an animated concept map concurrently with semantically equivalent spoken narration. The study used a 2 × 2 factorial design in which an animation factor (animated vs. static) was crossed with a representation factor (concept map vs. text). Students (N = 140) were randomly assigned to study one of four presentations on the human nervous system. The dependent measures were tests of free recall, knowledge and transfer. The concept map groups significantly outperformed the text groups on free recall and transfer. The animated concept map group did not significantly outperform the static map group. The authors hypothesize that the animated concept map provided no advantage over the static concept map because participants in both conditions were able to use the spoken narrative to sequence their reading.
Verbal redundancy arises from the concurrent presentation of text and verbatim speech. To inform ... more Verbal redundancy arises from the concurrent presentation of text and verbatim speech. To inform theories of multimedia learning that guide the design of educational materials, a meta-analysis was conducted to investigate the effects of spoken-only, written-only, and spoken–written presentations on learning retention and transfer. After an extensive search for experimental studies meeting specified inclusion criteria, data from 57 independent studies were extracted. Most of the research participants were postsecondary students. Overall, this meta-analysis revealed that outcomes comparing spoken–written and written-only presentations did not differ, but students who learned from spoken–written presentations outperformed those who learned from spoken-only presentations. This effect was dependent on learners' prior knowledge, pacing of presentation, and inclusion of animation or diagrams. Specifically, the advantages of spoken–written presentations over spoken-only presentations were found for low prior knowledge learners, system-paced learning materials, and picture-free materials. In comparison with verbatim, spoken–written presentations, presentations displaying key terms extracted from spoken narrations were associated with better learning outcomes and accounted for much of the advantage of spoken–written over spoken-only presentations. These findings have significant implications for the design of multimedia materials.
An animated concept map is a presentation of a network diagram in which nodes and links are seque... more An animated concept map is a presentation of a network diagram in which nodes and links are sequentially added or modified. An experiment compared learning from animated concept maps and text by randomly assigning 133 undergraduates to study 1 of 4 narrated animations presenting semantically equivalent information accompanied by identical audio narration. Two of the animations presented text; one with concurrent audio and another with delayed audio. Two of the animations presented concept maps; one in black and white and the other with nodes colored to represent semantic relatedness. The concept map groups outperformed the text groups on free recall (p < .05). The black-and-white concept map group outperformed the text groups on a multiple-choice knowledge test (p < .05). No advantages were statistically detected for color enhancements of the animated map. The results indicate that verbal information can be effectively communicated by learner-paced animated concept maps accompanied by audio narrations.
Educational psychology has generated a prolific array of findings about factors that influence an... more Educational psychology has generated a prolific array of findings about factors that influence and correlate with academic achievement. We review select findings from this voluminous literature and identify two domains of psychology: heuristics that describe generic relations between instructional designs and learning, which we call the psychology of “the way things are,” and findings about metacognition and self-regulated learning that demonstrate learners selectively apply and change their
use of those heuristics, which we call the psychology of “the way learners make things.” Distinguishing these domains highlights a need to marry two approaches to research methodology: the classical approach, which we describe as snapshot, bookend, between-group experimentation; and a microgenetic approach that traces proximal cause-effect bonds over time to validate theoretical accounts of how learning generates achievements. We argue for fusing these methods to advance a validated psychology of academic achievement.
Data were gathered from 248 secondary students (14 years old, 56% female) who learned about the c... more Data were gathered from 248 secondary students (14 years old, 56% female) who learned about the circulatory system in 77 self-selected collaborative groups. The learning outcomes of biology students who summarised by collaborative concept mapping were compared with those of students who summarised by collaborative writing. Learning groups randomly assigned to construct concept maps instead of conventional summaries generated more relations in the summary task and their members obtained higher individual scores on a post-test. The concept mapping strategy was found to be advantageous only for students whose cognitive ability was below the median for the sample and who were placed in groups with other students having low cognitive ability.
Handbook of research on collaborative learning using concept mapping, 2010
This chapter reviews research about the collaborative use of concept maps for learning. Although ... more This chapter reviews research about the collaborative use of concept maps for learning. Although decades of research have produced some understanding of the cognitive effects of using concept maps in individual learning, theories about their effects in collaborative learning are less firmly established. The review incorporated a systematic literature search, analysis of dependent variables as effect sizes, and discussion of representative studies. Students who learned collaboratively by constructing concept maps outperformed those who learned from other activities such as studying texts, outlines, lists and lectures. However, no effect of studying pre-constructed concept maps in collaborative settings was statistically detected. There was homogeneity across the subsets of studies investigated. The review concludes with suggestions for future research in learning with concept maps in collaborative environments.
The psychometric properties of scores from the Achievement Goal Questionnaire were examined in sa... more The psychometric properties of scores from the Achievement Goal Questionnaire were examined in samples of Japanese (N = 326) and Canadian ( N = 307) postsecondary students. Previous research found evidence of a four-factor structure of achievement goals in U. S. samples. Using confirmatory factor-analytic techniques, the authors found strong evidence for the four-factor structure of achievement goals in both the Canadian and Japanese populations. Subsequent multigroup structural equation modeling indicated the metric invariance of this four-factor structure across the two populations.
In evaluating a learning object with LORI, reviewers can rate and comment with respect to eight i... more In evaluating a learning object with LORI, reviewers can rate and comment with respect to eight items:
1. Content Quality: Accuracy, balanced presentation of ideas, appropriate level of detail, and reusability in varied contexts
2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics
3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling
4. Motivation: Ability to motivate and interest an identified population of learners
5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing
6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features
7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners
8. Standards Compliance: Adherence to international standards and operability on commonly used technical platforms
This exploratory case study examined in depth the studying activities of eight students across tw... more This exploratory case study examined in depth the studying activities of eight students across two studying episodes, and compared traces of actual studying activities to self-reports of self-regulated learning. Students participated in a 2-hour activity using our gStudy software to complete a course assignment. We used log file data to construct profiles of self-regulated learning activity in four ways: (a) frequency of studying events, (b) patterns of studying activity, (c) timing and sequencing of events, and (d) content analyses of students' notes and summaries. Findings indicate that students' self-reports may not calibrate to actual studying activity. Analyses of log file traces of studying activities provide important information for defining strategies and sequences of fine-grained studying actions. We contrast these analytic methods and illustrate how trace-based profiles of students' self-regulated studying inform models of metacognitive monitoring, evaluation, and self-regulated adaptation.
This article presents the structure and theoretical foundations of the Learning Object Review Ins... more This article presents the structure and theoretical foundations of the Learning Object Review Instrument (LORI), an evaluation aid available through the E-Learning Research and Assessment Network at http://www.elera.net. A primary goal of LORI is to balance assessment validity with efficiency of the evaluation process. The instrument enables learning object users to create reviews consisting of ratings and comments on nine dimensions of quality: content quality, learning goal alignment, feedback and adaptation, motivation, presentation design, interaction usability, accessibility, reusability, and standards compliance. The article presents research and practices relevant to these dimensions and describes how each dimension can be interpreted to evaluate multimedia learning resources.
We have developed a software application (gStudy) that supports learning with multimedia document... more We have developed a software application (gStudy) that supports learning with multimedia documents. Students use gStudy to create and link notes, highlight and label text and images, construct glossaries and concept maps, exchange information objects through a chat interface, and perform other operations on multimedia content. The detailed log files recorded by gStudy constitute a wealth of data about how students process information as they learn. In this paper we describe log parsing and data mining methods we have borrowed from computer science and apply to researching self-regulated learning. The analysis software we developed identifies coherent learner actions from the complex series of low-level events recorded by gStudy and detects sequential patterns of these actions that may be interspersed with unrelated actions. It applies a data mining algorithm to discover action patterns which are the longest subsequences common to a group of participants. The use of these methods is illustrated through an analysis of gStudy log files generated by 103 university students.
E‐learning networked environments and architectures: A knowledge processing perspective, 2007
The unceasing growth of the Internet has led to new modes of learning in which learners routinely... more The unceasing growth of the Internet has led to new modes of learning in which learners routinely interact on-line with instructors, other students, and digital resources. Much recent research has focused on building infrastructure for these activities, especially to facilitate searching, filtering, and recommending on-line resources known as learning objects. Although newly defined standards for learning object metadata are expected to greatly improve searching and filtering capabilities, learners, instructors, and instructional developers may still be faced with choosing from many pages of object listings returned from a single learning object query. The listed objects tend to vary widely in quality. With current metadata and search methods, those who search for learning objects waste time and effort groping through overwhelming masses of information, often finding only poorly designed and developed instructional materials. Hence, there is a clear need for quality evaluations prior to making a recommendation that can be communicated in a coherent, standardized format to measure the quality of learning objects.
Concept maps are spatial diagrams showing interrelationships among concepts as node-link assembli... more Concept maps are spatial diagrams showing interrelationships among concepts as node-link assemblies. In this report we use the term concept map to refer to knowledge maps, semantic networks and similar representations. Since the early 1980s, researchers have demonstrated increasing interest in the instructional use of concept maps (Novak & Gowin, 1984; O’Donnell, Dansereau & Hall, 2002). A recent meta-analysis by Nesbit and Adesope (2006) found that studying with concept maps offered an average learning effect of about .4 standard deviations when compared to studying with texts. The learning gain from concept maps is even more apparent when students construct their own concept maps or when they modify teachers’ maps. Despite the growing importance of concept maps as instructional tools, no research has collected eye movement data to investigate how learners read concept maps, and how their graphical properties affect comprehension and learning. The research reported here investigates the order in which nodes are processed as learners begin the task of interpreting the meaning of a concept map.
This chapter surveys a landscape of issues, trends and practices in the methods of contemporary e... more This chapter surveys a landscape of issues, trends and practices in the methods of contemporary educational psychology research. The topics selected are those we believe to have significant implications for the future of quantitatively oriented research. Of course, space limitations do not permit an exhaustive survey of this wide domain, and indeed, we were unable to include some developments that anticipate future methodological practices. We believe that many of the methods reviewed in this chapter can contribute to new research perspectives that advance "beyond the quantitative-qualitative divide". The innovations and trends that we sample originate from a variety of sources. Some are driven by methodological advances in psychology, including methods for tracking cognitive load and the trend toward reporting effect sizes and confidence intervals. Other trends, including the use of interactive learning environments, log file analysis, and online data gathering, are driven by the availability of new technologies. Some methodological issues, such as the use of portfolios and concept maps for research assessment, are guided by developments in educational practice and other areas of educational research. Finally, methodological advances such as techniques for tracking shifts in students' interest as they read, are originating within educational psychology as researchers strive to adapt methods to match evolving theories. Chapter subtitles include the following: matching the method to the matter: issues of grain size; advances in assessing psychological constructs; alternatives for learner assessment in educational research; advances in technology-mediated research; trends in reporting research outcomes; and finally, emerging themes.
This meta-analysis reviews experimental and quasi-experimental studies in which students learned ... more This meta-analysis reviews experimental and quasi-experimental studies in which students learned by constructing, modifying, or viewing node-link diagrams. Following an exhaustive search for studies meeting specified design criteria, 67 standardized mean difference effect sizes were extracted from 55 studies involving 5,818 participants. Students at levels ranging from Grade 4 to postsecondary used concept maps to learn in domains such as science, psychology, statistics, and nursing. Posttests measured recall and transfer. Across several instructional conditions, settings, and methodological features, the use of concept maps was associated with increased knowledge retention. Mean effect sizes varied from small to large depending on how concept maps were used and on the type of comparison treatment. Significant heterogeneity was found in most subsets.
2014 IEEE 14th International Conference on Advanced Learning Technologies
A meta-analysis on the effectiveness of Intelligent Tutoring Systems (ITS) in computer science ed... more A meta-analysis on the effectiveness of Intelligent Tutoring Systems (ITS) in computer science education compared the learning outcomes of ITS and non-ITS instruction. A search of the literature found 22 effect sizes (involving 1,447 participants) that met the pre-defined inclusion criteria. Although most of the ITS were used to teach programming, other topics such as database design and computer literacy were also represented. There was a significant overall effect size favoring the use of ITS. There was a significant advantage of ITS over teacher-led classroom instruction and non-ITS computer-based instruction. ITS were more effective than the instructional methods to which they were compared regardless of whether they modeled misconceptions and regardless of whether they were the primary means of instruction or were an integrated component of learning activities that included other means of instruction.
Intelligent Tutoring Systems (ITS) are computer programs that model learners’ psychological state... more Intelligent Tutoring Systems (ITS) are computer programs that model learners’ psychological states to provide individualized instruction. They have been developed for diverse subject areas (e.g., algebra, medicine, law, reading) to help learners acquire domain-specific, cognitive and metacognitive knowledge. A meta-analysis was conducted on research that compared the outcomes from students learning from ITS to those learning from non-ITS learning environments. The meta-analysis examined how effect sizes varied with type of ITS, type of comparison treatment received by learners, type of learning outcome, whether knowledge to be learned was procedural or declarative, and other factors. After a search of major bibliographic databases, 107 effect sizes involving 14,321 participants were extracted and analyzed. The use of ITS was associated with greater achievement in comparison with teacher-led, large-group instruction (g = .42), non-ITS computer-based instruction (g = .57), and textbooks or workbooks (g = .35). There was no significant difference between learning from ITS and learning from individualized human tutoring (g = –.11) or small-group instruction (g = .05). Significant, positive mean effect sizes were found regardless of whether the ITS was used as the principal means of instruction, a supplement to teacher-led instruction, an integral component of teacher-led instruction, or an aid to homework. Significant, positive effect sizes were found at all levels of education, in almost all subject domains evaluated, and whether or not the ITS provided feedback or modeled student misconceptions. The claim that ITS are relatively effective tools for learning is consistent with our analysis of potential publication bias.
Proceedings of the 14th European Workshop on Natural Language Generation, 2013
When instructors prepare learning materials for students, they frequently develop accompanying qu... more When instructors prepare learning materials for students, they frequently develop accompanying questions to guide learning. Natural language processing technology can be used to automatically generate such questions but techniques used have not fully leveraged semantic information contained in the learning materials or the full context in which the question generation task occurs. We introduce a sophisticated template-based approach that incorporates semantic role labels into a system that automatically generates natural language questions to support online learning. While we have not yet incorporated the full learning context into our approach, our preliminary evaluation and evaluation methodology indicate our approach is a promising one for supporting learning.
An animated concept map represents verbal information in a node-link diagram that changes over ti... more An animated concept map represents verbal information in a node-link diagram that changes over time. The goals of the experiment were to evaluate the instructional effects of presenting an animated concept map concurrently with semantically equivalent spoken narration. The study used a 2 × 2 factorial design in which an animation factor (animated vs. static) was crossed with a representation factor (concept map vs. text). Students (N = 140) were randomly assigned to study one of four presentations on the human nervous system. The dependent measures were tests of free recall, knowledge and transfer. The concept map groups significantly outperformed the text groups on free recall and transfer. The animated concept map group did not significantly outperform the static map group. The authors hypothesize that the animated concept map provided no advantage over the static concept map because participants in both conditions were able to use the spoken narrative to sequence their reading.
Verbal redundancy arises from the concurrent presentation of text and verbatim speech. To inform ... more Verbal redundancy arises from the concurrent presentation of text and verbatim speech. To inform theories of multimedia learning that guide the design of educational materials, a meta-analysis was conducted to investigate the effects of spoken-only, written-only, and spoken–written presentations on learning retention and transfer. After an extensive search for experimental studies meeting specified inclusion criteria, data from 57 independent studies were extracted. Most of the research participants were postsecondary students. Overall, this meta-analysis revealed that outcomes comparing spoken–written and written-only presentations did not differ, but students who learned from spoken–written presentations outperformed those who learned from spoken-only presentations. This effect was dependent on learners' prior knowledge, pacing of presentation, and inclusion of animation or diagrams. Specifically, the advantages of spoken–written presentations over spoken-only presentations were found for low prior knowledge learners, system-paced learning materials, and picture-free materials. In comparison with verbatim, spoken–written presentations, presentations displaying key terms extracted from spoken narrations were associated with better learning outcomes and accounted for much of the advantage of spoken–written over spoken-only presentations. These findings have significant implications for the design of multimedia materials.
An animated concept map is a presentation of a network diagram in which nodes and links are seque... more An animated concept map is a presentation of a network diagram in which nodes and links are sequentially added or modified. An experiment compared learning from animated concept maps and text by randomly assigning 133 undergraduates to study 1 of 4 narrated animations presenting semantically equivalent information accompanied by identical audio narration. Two of the animations presented text; one with concurrent audio and another with delayed audio. Two of the animations presented concept maps; one in black and white and the other with nodes colored to represent semantic relatedness. The concept map groups outperformed the text groups on free recall (p < .05). The black-and-white concept map group outperformed the text groups on a multiple-choice knowledge test (p < .05). No advantages were statistically detected for color enhancements of the animated map. The results indicate that verbal information can be effectively communicated by learner-paced animated concept maps accompanied by audio narrations.
Educational psychology has generated a prolific array of findings about factors that influence an... more Educational psychology has generated a prolific array of findings about factors that influence and correlate with academic achievement. We review select findings from this voluminous literature and identify two domains of psychology: heuristics that describe generic relations between instructional designs and learning, which we call the psychology of “the way things are,” and findings about metacognition and self-regulated learning that demonstrate learners selectively apply and change their
use of those heuristics, which we call the psychology of “the way learners make things.” Distinguishing these domains highlights a need to marry two approaches to research methodology: the classical approach, which we describe as snapshot, bookend, between-group experimentation; and a microgenetic approach that traces proximal cause-effect bonds over time to validate theoretical accounts of how learning generates achievements. We argue for fusing these methods to advance a validated psychology of academic achievement.
Data were gathered from 248 secondary students (14 years old, 56% female) who learned about the c... more Data were gathered from 248 secondary students (14 years old, 56% female) who learned about the circulatory system in 77 self-selected collaborative groups. The learning outcomes of biology students who summarised by collaborative concept mapping were compared with those of students who summarised by collaborative writing. Learning groups randomly assigned to construct concept maps instead of conventional summaries generated more relations in the summary task and their members obtained higher individual scores on a post-test. The concept mapping strategy was found to be advantageous only for students whose cognitive ability was below the median for the sample and who were placed in groups with other students having low cognitive ability.
Handbook of research on collaborative learning using concept mapping, 2010
This chapter reviews research about the collaborative use of concept maps for learning. Although ... more This chapter reviews research about the collaborative use of concept maps for learning. Although decades of research have produced some understanding of the cognitive effects of using concept maps in individual learning, theories about their effects in collaborative learning are less firmly established. The review incorporated a systematic literature search, analysis of dependent variables as effect sizes, and discussion of representative studies. Students who learned collaboratively by constructing concept maps outperformed those who learned from other activities such as studying texts, outlines, lists and lectures. However, no effect of studying pre-constructed concept maps in collaborative settings was statistically detected. There was homogeneity across the subsets of studies investigated. The review concludes with suggestions for future research in learning with concept maps in collaborative environments.
The psychometric properties of scores from the Achievement Goal Questionnaire were examined in sa... more The psychometric properties of scores from the Achievement Goal Questionnaire were examined in samples of Japanese (N = 326) and Canadian ( N = 307) postsecondary students. Previous research found evidence of a four-factor structure of achievement goals in U. S. samples. Using confirmatory factor-analytic techniques, the authors found strong evidence for the four-factor structure of achievement goals in both the Canadian and Japanese populations. Subsequent multigroup structural equation modeling indicated the metric invariance of this four-factor structure across the two populations.
In evaluating a learning object with LORI, reviewers can rate and comment with respect to eight i... more In evaluating a learning object with LORI, reviewers can rate and comment with respect to eight items:
1. Content Quality: Accuracy, balanced presentation of ideas, appropriate level of detail, and reusability in varied contexts
2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics
3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling
4. Motivation: Ability to motivate and interest an identified population of learners
5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing
6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features
7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners
8. Standards Compliance: Adherence to international standards and operability on commonly used technical platforms
This exploratory case study examined in depth the studying activities of eight students across tw... more This exploratory case study examined in depth the studying activities of eight students across two studying episodes, and compared traces of actual studying activities to self-reports of self-regulated learning. Students participated in a 2-hour activity using our gStudy software to complete a course assignment. We used log file data to construct profiles of self-regulated learning activity in four ways: (a) frequency of studying events, (b) patterns of studying activity, (c) timing and sequencing of events, and (d) content analyses of students' notes and summaries. Findings indicate that students' self-reports may not calibrate to actual studying activity. Analyses of log file traces of studying activities provide important information for defining strategies and sequences of fine-grained studying actions. We contrast these analytic methods and illustrate how trace-based profiles of students' self-regulated studying inform models of metacognitive monitoring, evaluation, and self-regulated adaptation.
This article presents the structure and theoretical foundations of the Learning Object Review Ins... more This article presents the structure and theoretical foundations of the Learning Object Review Instrument (LORI), an evaluation aid available through the E-Learning Research and Assessment Network at http://www.elera.net. A primary goal of LORI is to balance assessment validity with efficiency of the evaluation process. The instrument enables learning object users to create reviews consisting of ratings and comments on nine dimensions of quality: content quality, learning goal alignment, feedback and adaptation, motivation, presentation design, interaction usability, accessibility, reusability, and standards compliance. The article presents research and practices relevant to these dimensions and describes how each dimension can be interpreted to evaluate multimedia learning resources.
We have developed a software application (gStudy) that supports learning with multimedia document... more We have developed a software application (gStudy) that supports learning with multimedia documents. Students use gStudy to create and link notes, highlight and label text and images, construct glossaries and concept maps, exchange information objects through a chat interface, and perform other operations on multimedia content. The detailed log files recorded by gStudy constitute a wealth of data about how students process information as they learn. In this paper we describe log parsing and data mining methods we have borrowed from computer science and apply to researching self-regulated learning. The analysis software we developed identifies coherent learner actions from the complex series of low-level events recorded by gStudy and detects sequential patterns of these actions that may be interspersed with unrelated actions. It applies a data mining algorithm to discover action patterns which are the longest subsequences common to a group of participants. The use of these methods is illustrated through an analysis of gStudy log files generated by 103 university students.
E‐learning networked environments and architectures: A knowledge processing perspective, 2007
The unceasing growth of the Internet has led to new modes of learning in which learners routinely... more The unceasing growth of the Internet has led to new modes of learning in which learners routinely interact on-line with instructors, other students, and digital resources. Much recent research has focused on building infrastructure for these activities, especially to facilitate searching, filtering, and recommending on-line resources known as learning objects. Although newly defined standards for learning object metadata are expected to greatly improve searching and filtering capabilities, learners, instructors, and instructional developers may still be faced with choosing from many pages of object listings returned from a single learning object query. The listed objects tend to vary widely in quality. With current metadata and search methods, those who search for learning objects waste time and effort groping through overwhelming masses of information, often finding only poorly designed and developed instructional materials. Hence, there is a clear need for quality evaluations prior to making a recommendation that can be communicated in a coherent, standardized format to measure the quality of learning objects.
Concept maps are spatial diagrams showing interrelationships among concepts as node-link assembli... more Concept maps are spatial diagrams showing interrelationships among concepts as node-link assemblies. In this report we use the term concept map to refer to knowledge maps, semantic networks and similar representations. Since the early 1980s, researchers have demonstrated increasing interest in the instructional use of concept maps (Novak & Gowin, 1984; O’Donnell, Dansereau & Hall, 2002). A recent meta-analysis by Nesbit and Adesope (2006) found that studying with concept maps offered an average learning effect of about .4 standard deviations when compared to studying with texts. The learning gain from concept maps is even more apparent when students construct their own concept maps or when they modify teachers’ maps. Despite the growing importance of concept maps as instructional tools, no research has collected eye movement data to investigate how learners read concept maps, and how their graphical properties affect comprehension and learning. The research reported here investigates the order in which nodes are processed as learners begin the task of interpreting the meaning of a concept map.
This chapter surveys a landscape of issues, trends and practices in the methods of contemporary e... more This chapter surveys a landscape of issues, trends and practices in the methods of contemporary educational psychology research. The topics selected are those we believe to have significant implications for the future of quantitatively oriented research. Of course, space limitations do not permit an exhaustive survey of this wide domain, and indeed, we were unable to include some developments that anticipate future methodological practices. We believe that many of the methods reviewed in this chapter can contribute to new research perspectives that advance "beyond the quantitative-qualitative divide". The innovations and trends that we sample originate from a variety of sources. Some are driven by methodological advances in psychology, including methods for tracking cognitive load and the trend toward reporting effect sizes and confidence intervals. Other trends, including the use of interactive learning environments, log file analysis, and online data gathering, are driven by the availability of new technologies. Some methodological issues, such as the use of portfolios and concept maps for research assessment, are guided by developments in educational practice and other areas of educational research. Finally, methodological advances such as techniques for tracking shifts in students' interest as they read, are originating within educational psychology as researchers strive to adapt methods to match evolving theories. Chapter subtitles include the following: matching the method to the matter: issues of grain size; advances in assessing psychological constructs; alternatives for learner assessment in educational research; advances in technology-mediated research; trends in reporting research outcomes; and finally, emerging themes.
This meta-analysis reviews experimental and quasi-experimental studies in which students learned ... more This meta-analysis reviews experimental and quasi-experimental studies in which students learned by constructing, modifying, or viewing node-link diagrams. Following an exhaustive search for studies meeting specified design criteria, 67 standardized mean difference effect sizes were extracted from 55 studies involving 5,818 participants. Students at levels ranging from Grade 4 to postsecondary used concept maps to learn in domains such as science, psychology, statistics, and nursing. Posttests measured recall and transfer. Across several instructional conditions, settings, and methodological features, the use of concept maps was associated with increased knowledge retention. Mean effect sizes varied from small to large depending on how concept maps were used and on the type of comparison treatment. Significant heterogeneity was found in most subsets.
Uploads
Papers by John Nesbit
use of those heuristics, which we call the psychology of “the way learners make things.” Distinguishing these domains highlights a need to marry two approaches to research methodology: the classical approach, which we describe as snapshot, bookend, between-group experimentation; and a microgenetic approach that traces proximal cause-effect bonds over time to validate theoretical accounts of how learning generates achievements. We argue for fusing these methods to advance a validated psychology of academic achievement.
1. Content Quality: Accuracy, balanced presentation of ideas, appropriate level of detail, and reusability in varied contexts
2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics
3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling
4. Motivation: Ability to motivate and interest an identified population of learners
5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing
6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features
7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners
8. Standards Compliance: Adherence to international standards and operability on commonly used technical platforms
use of those heuristics, which we call the psychology of “the way learners make things.” Distinguishing these domains highlights a need to marry two approaches to research methodology: the classical approach, which we describe as snapshot, bookend, between-group experimentation; and a microgenetic approach that traces proximal cause-effect bonds over time to validate theoretical accounts of how learning generates achievements. We argue for fusing these methods to advance a validated psychology of academic achievement.
1. Content Quality: Accuracy, balanced presentation of ideas, appropriate level of detail, and reusability in varied contexts
2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics
3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling
4. Motivation: Ability to motivate and interest an identified population of learners
5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing
6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features
7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners
8. Standards Compliance: Adherence to international standards and operability on commonly used technical platforms