Conversational discourse is a cognitive and social process influenced by both discourse content a... more Conversational discourse is a cognitive and social process influenced by both discourse content and pragmatic factors, such as the participants’ prior knowledge; these factors may also affect how simulated conversations with virtual agents unfold, with implications for design. This study explored effects of question content and perceived expertise of a virtual agent on students’ interactions with a conversation-based assessment (CBA) measuring science inquiry skills. Twenty-four middle school students were randomly assigned to work with a Highor Low-Knowledge virtual peer to collect data and generate weather predictions. Students evaluated their own data relative to the peer’s; they could either ”Choose” which note to keep, or to ”Agree/Disagree” with the peer’s suggested choice of note. Students rated the peer as more expert in the High-Knowledge condition, but peer expertise did not affect performance. However, the Agree/Disagree condition improved students’ accuracy in their note...
Despite recent advances in technology, personalized learning to address diverse needs of students... more Despite recent advances in technology, personalized learning to address diverse needs of students remains difficult to achieve at scale. With the availability and affordability of smart devices in the era of the Internet of Things, learners, parents, and educators are more “connected” than ever before. Education stakeholders and technology developers can leverage these advances to collect data about, inform, deliver, and improve education for all learners. In this paper, we review the core components of a Smart Learning framework and describe a personalized mastery-based learning system that leverages the framework to deliver personalized learning at scale. In the context of Smart Learning in the Internet of Things, we propose an Ambient and Pervasive Personalized Learning Ecosystem (APPLE), a learner-centered approach that uses Bloom’s Four Agents of Change in the Internet of Things ecosystem to provide learners a comprehensive and personalized learning experience. This ecosystem u...
Literacy Research: Theory, Method, and Practice, 2019
Multiple-source inquiry that involves collaboration and deliberation is a complex construct; acco... more Multiple-source inquiry that involves collaboration and deliberation is a complex construct; accordingly, valid measurement of these related competencies should require students to demonstrate their collaborative inquiry and social deliberation skills as part of an integrated performance. In this article, we report findings from the first phase of an assessment project designed to develop and evaluate the use of a virtual world platform that simultaneously captures collaboration, online inquiry, and social deliberation in the context of a single scenario-based task. Mixed methods were employed to evaluate preliminary evidence of students’ (ages 14–16) competencies in both face-to-face ( n = 5 dyads) and remote ( n = 6 dyads) contexts. Results indicated the scenario-based virtual assessment platform successfully captured varying levels of cognitive and social performance (using both product and process data) on several dimensions of collaborative online inquiry and social deliberatio...
We describe efforts to design and validate a digitally-based assessment of collaborative online i... more We describe efforts to design and validate a digitally-based assessment of collaborative online inquiry and social deliberation using a digital virtual world platform with embedded supports for real-time collaboration. Cognitive validity studies were conducted to examine the collaborative prompts and the overall task, with 21 dyads participating in either Face-to-Face (FTF; n=5) or Computer-Mediated (CM; n=16) conditions. Quantitative results suggest the task captured variation in dyads’ inquiry performances and processes. Assessment of 21 century skills: Online inquiry and collaboration Critical thinking, complex problem solving, and collaboration are required 21 century skills for success in college and the workforce. Current large-scale assessments in the United States do not fully represent the range of 21 century skills associated with the complexities of digital literacy and information-based problem solving in collaborative, networked environments; thus, there has been increa...
Literacy Research: Theory, Method, and Practice, 2019
Multiple-source inquiry that involves collaboration and deliberation is a complex construct; acco... more Multiple-source inquiry that involves collaboration and deliberation is a complex construct; accordingly, valid measurement of these related competencies should require students to demonstrate their collaborative inquiry and social deliberation skills as part of an integrated performance. In this article, we report findings from the first phase of an assessment project designed to develop and evaluate the use of a virtual world platform that simultaneously captures collaboration, online inquiry, and social deliberation in the context of a single scenario-based task. Mixed methods were employed to evaluate preliminary evidence of students' (ages 14-16) competencies in both face-to-face (n=5 dyads) and remote (n=6 dyads) contexts. Results indicated the scenario-based virtual assessment platform successfully captured varying levels of cognitive and social performance (using both product and process data) on several dimensions of collaborative online inquiry and social deliberation. Illustrative examples of dyads demonstrating stronger or weaker collaborative processes and high-level themes that emerged are presented. Implications for classroom settings and large scale assessment environments are discussed.
The purpose of this chapter is three-fold. First, we synthesize key theories and research that in... more The purpose of this chapter is three-fold. First, we synthesize key theories and research that inform our vision of how online collaborative inquiry and social deliberation work together in digital learning environments. Second, we review current efforts to assess these constructs, including details from our emerging work under the National Assessment of Educational Progress (NAEP) Survey Assessment Innovations Laboratory (SAIL) Initiative, which is dedicated to exploring potential innovations in large-scale assessment by leveraging cognitive science research and new technologies. Finally, we outline key challenges associated with work in this area as well as implications for educational research and practice.
Conversational discourse is a cognitive and social process influenced by both discourse content a... more Conversational discourse is a cognitive and social process influenced by both discourse content and pragmatic factors, such as the participants’ prior knowledge; these factors may also affect how simulated conversations with virtual agents unfold, with implications for design. This study explored effects of question content and perceived expertise of a virtual agent on students’ interactions with a conversation-based assessment (CBA) measuring science inquiry skills. Twenty-four middle school students were randomly assigned to work with a Highor Low-Knowledge virtual peer to collect data and generate weather predictions. Students evaluated their own data relative to the peer’s; they could either ”Choose” which note to keep, or to ”Agree/Disagree” with the peer’s suggested choice of note. Students rated the peer as more expert in the High-Knowledge condition, but peer expertise did not affect performance. However, the Agree/Disagree condition improved students’ accuracy in their note...
Despite recent advances in technology, personalized learning to address diverse needs of students... more Despite recent advances in technology, personalized learning to address diverse needs of students remains difficult to achieve at scale. With the availability and affordability of smart devices in the era of the Internet of Things, learners, parents, and educators are more “connected” than ever before. Education stakeholders and technology developers can leverage these advances to collect data about, inform, deliver, and improve education for all learners. In this paper, we review the core components of a Smart Learning framework and describe a personalized mastery-based learning system that leverages the framework to deliver personalized learning at scale. In the context of Smart Learning in the Internet of Things, we propose an Ambient and Pervasive Personalized Learning Ecosystem (APPLE), a learner-centered approach that uses Bloom’s Four Agents of Change in the Internet of Things ecosystem to provide learners a comprehensive and personalized learning experience. This ecosystem u...
Literacy Research: Theory, Method, and Practice, 2019
Multiple-source inquiry that involves collaboration and deliberation is a complex construct; acco... more Multiple-source inquiry that involves collaboration and deliberation is a complex construct; accordingly, valid measurement of these related competencies should require students to demonstrate their collaborative inquiry and social deliberation skills as part of an integrated performance. In this article, we report findings from the first phase of an assessment project designed to develop and evaluate the use of a virtual world platform that simultaneously captures collaboration, online inquiry, and social deliberation in the context of a single scenario-based task. Mixed methods were employed to evaluate preliminary evidence of students’ (ages 14–16) competencies in both face-to-face ( n = 5 dyads) and remote ( n = 6 dyads) contexts. Results indicated the scenario-based virtual assessment platform successfully captured varying levels of cognitive and social performance (using both product and process data) on several dimensions of collaborative online inquiry and social deliberatio...
We describe efforts to design and validate a digitally-based assessment of collaborative online i... more We describe efforts to design and validate a digitally-based assessment of collaborative online inquiry and social deliberation using a digital virtual world platform with embedded supports for real-time collaboration. Cognitive validity studies were conducted to examine the collaborative prompts and the overall task, with 21 dyads participating in either Face-to-Face (FTF; n=5) or Computer-Mediated (CM; n=16) conditions. Quantitative results suggest the task captured variation in dyads’ inquiry performances and processes. Assessment of 21 century skills: Online inquiry and collaboration Critical thinking, complex problem solving, and collaboration are required 21 century skills for success in college and the workforce. Current large-scale assessments in the United States do not fully represent the range of 21 century skills associated with the complexities of digital literacy and information-based problem solving in collaborative, networked environments; thus, there has been increa...
Literacy Research: Theory, Method, and Practice, 2019
Multiple-source inquiry that involves collaboration and deliberation is a complex construct; acco... more Multiple-source inquiry that involves collaboration and deliberation is a complex construct; accordingly, valid measurement of these related competencies should require students to demonstrate their collaborative inquiry and social deliberation skills as part of an integrated performance. In this article, we report findings from the first phase of an assessment project designed to develop and evaluate the use of a virtual world platform that simultaneously captures collaboration, online inquiry, and social deliberation in the context of a single scenario-based task. Mixed methods were employed to evaluate preliminary evidence of students' (ages 14-16) competencies in both face-to-face (n=5 dyads) and remote (n=6 dyads) contexts. Results indicated the scenario-based virtual assessment platform successfully captured varying levels of cognitive and social performance (using both product and process data) on several dimensions of collaborative online inquiry and social deliberation. Illustrative examples of dyads demonstrating stronger or weaker collaborative processes and high-level themes that emerged are presented. Implications for classroom settings and large scale assessment environments are discussed.
The purpose of this chapter is three-fold. First, we synthesize key theories and research that in... more The purpose of this chapter is three-fold. First, we synthesize key theories and research that inform our vision of how online collaborative inquiry and social deliberation work together in digital learning environments. Second, we review current efforts to assess these constructs, including details from our emerging work under the National Assessment of Educational Progress (NAEP) Survey Assessment Innovations Laboratory (SAIL) Initiative, which is dedicated to exploring potential innovations in large-scale assessment by leveraging cognitive science research and new technologies. Finally, we outline key challenges associated with work in this area as well as implications for educational research and practice.
Uploads
Papers by Jesse R Sparks
assessment environments are discussed.
assessment environments are discussed.