The Potential of Learning With AI-Generated Pedagogical Agents in Instructional Videos
Author: 
Jullia LimAuthors Info & Claims
Jullia LimAuthors Info & ClaimsArticle No.: 615, Pages 1 - 6
Abstract
With the recent advancement in technology, generative artificial intelligence (GenAI) can produce hyper-realistic multimedia content, such as audio, text, images, and videos. Although this technology has raised great concerns about its misuse and harmful applications, it holds great potential to revolutionize traditional ways of teaching and learning. The use of GenAI in education has increased markedly, however, pedagogical research on this rapidly emerging technology is yet to be studied extensively. There is an urgent need to investigate the unexamined potential of this technology. Therefore, this ongoing research will explore the potential of AI-generated pedagogical agents (PA), or avatars, in instructional videos to facilitate learning. The effects of the type of PA (AI-generated, real-life human), and voice (AI-generated, human voice) on an individual's learning outcomes, cognitive load, motivation, and attention will be studied. Findings from a pilot study provide some preliminary evidence that PA appearance influences learners’ retention and cognitive load, but not attention. The type of PA influenced learners' perception of the agent's ability to facilitate learning, its human-like qualities, and its engagement level. However, it did not affect its credibility. This ongoing work will contribute to the growing understanding of the impact of AI in education, provide evidence of the efficacy of AI-generated PAs in instructional videos for learning, and narrow the gap between human-computer interaction research and education.
References
[1]
Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2), 416–427. https://doi.org/10.1037/0022-0663.94.2.416
[2]
Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in multimedia learning: Examining the impact of an animated agent's voice. Contemporary Educational Psychology, 30(1), 117–139. https://doi.org/10.1016/j.cedpsych.2004.07.001
[3]
Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450–457. https://doi.org/10.1016/j.chb.2008.10.008
[4]
Baylor, A. L., & Kim, Y. (2004). Pedagogical agent design: The impact of agent realism, gender, ethnicity, and instructional role. Intelligent Tutoring Systems, 592–603. https://doi.org/10.1007/978-3-540-30139-4_56
[5]
Baylor, A. L., & Ryu, J. (2003). The API (Agent Persona Instrument) for assessing pedagogical agent persona. In D. Lassner & C. McNaught (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003 (pp. 448-451). Chesapeake, VA: AACE
[6]
Bellini, S., & Akullian, J. (2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional Children, 73, 261–284. https://doi.org/10.1177/001440290707300301.
[7]
Bosseler, A., & Massaro, D. W. (2003). Development and evaluation of a computer-animated tutor for vocabulary and language learning in children with autism. Journal of Autism and Developmental Disorders, 33, 653–673. https://doi.org/10.1023/B:JADD.0000006002.82367.4f.
[8]
Chan, C. K., & Hu, W. (2023). Students’ voices on Generative AI: Perceptions, benefits, and challenges in Higher Education. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00411-8
[9]
Charlton, C. T., Kellems, R. O., Black, B., Bussey, H. C., Ferguson, R., Goncalves, B., Jensen, M., & Vallejo, S. (2020). Effectiveness of avatar-delivered instruction on social initiations by children with autism spectrum disorder. Research in Autism Spectrum Disorders, 71, 101494. https://doi.org/10.1016/j.rasd.2019.101494
[10]
Clarebout, G., & Heidig (née Domagk), S. (2012). Pedagogical Agents. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_942
[11]
Dai, L., Jung, M. M., Postma, M., & Louwerse, M. M. (2022). A systematic review of Pedagogical Agent Research: Similarities, differences and unexplored aspects. Computers & Education, 190, 104607. https://doi.org/10.1016/j.compedu.2022.104607
[12]
Dunsworth, Q., & Atkinson, R. K. (2007). Fostering multimedia learning of science: Explaining the role of an animated agent's image. Computers & Education, 49, 677–690.
[13]
Erdodi, L.A., Lajiness-O'Neill, R., & Schmitt, T.A. (2013). Learning curve analyses in neurodevelopmental disorders: Are children with autism spectrum disorder truly visual learners? Journal of Autism and Developmental Disorders, 43, 880–890.
[14]
Fiorella, L., & Mayer, R. (2021). Principles Based on Social Cues in Multimedia Learning: Personalization, Voice, Image, and Embodiment Principles. In R. Mayer & L. Fiorella (Eds.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 277-285). Cambridge: Cambridge University Press.
[15]
Keller, J.M. (2010). Motivational Design for Learning and Performance: The ARCS Model Approach. New York, NY: Springer.
[16]
Leiker, D., & Gyllen, A. R., Eldesouky, I. and Cukurova, M. (2023). Generative AI for learning: Investigating the potential of synthetic learning videos. Proceedings of International Conference of Artificial Intelligence in Education. Industrial Management & Data Systems, 115(5), 833–852.
[17]
Leppink, J., Paas, F., Van der Vleuten, C. M., Van Gog, T. & Van Merriënboer, J. G. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058-1072.
[18]
Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M. & van Merriënboer, J. J. G. (2014). Effects of pairs of problems and examples on task performance and different types of cognitive load. Learning and Instruction, 30(0), 32-42.
[19]
Lim, W.M., Gunasekara, A., Pallant, J.L., Pallant, J.L., & Pechenkina, E. (2023). Generative ai and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. International Journal of Management in Education, 21(2), Article 100790, https://doi.org/10.1016/j.ijme.2023.100790
[20]
Mayer, R. E. (2014). Principles based on social cues in multimedia learning: personalization, voice, image, and embodiment principles. In Cambridge University Press eBooks (pp. 345–368). https://doi.org/10.1017/cbo9781139547369.017
[21]
Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93, 390–397.
[22]
Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with animated pedagogical agents. Journal of Experimental Psychology: Applied, 18(3), 239–252. https://doi.org/10.1037/a0028616
[23]
Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: Role of speaker's voice. Journal of Educational Psychology, 95(2), 419–425. https://doi.org/10.1037/0022-0663.95.2.419
[24]
Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social Agency in Computer-Based Teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177–213. https://doi.org/10.1207/s1532690xci1902_02
[25]
Moreno, R., Reislein, M., & Özoğul, G. (2010). Using virtual peers to guide visual attention during learning. Journal of Media Psychology, 22(2), 52–60. https://doi.org/10.1027/1864-1105/a000008
[26]
Rusandi, M.A., Saripah, A.I., Khairun, D.Y., & M. (2023). No worries with ChatGPT: building bridges between artificial intelligence and education with critical thinking soft skills. Journal of Public Health, https://doi.org/10.1093/ /fdad049
[27]
Shimada, S. (2010). Deactivation in the sensorimotor area during observation of a human agent performing robotic actions. Brain and Cognition, 72(3), 394–399. https://doi.org/10.1016/j.bandc.2009.11.005.
[28]
Su, J., Ng, D. T. K., & Chu, S. K. W. (2023). Artificial intelligence (AI) literacy in early childhood education: The challenges and opportunities. Computers and Education: Artificial Intelligence, 4, 100124. https://doi.org/10.1016/j.caeai.2023.100124
[29]
Taecharungroj, V. (2023). “What can ChatGPT do”? Analyzing early reaction to the innovative AI chatbot on Twitter. Big Data and Cognitive Computing, 7(35), https://doi.org/10.3390/bdcc7010035
[30]
Wang, F., Li, W., Xie, H., & Liu, H. (2017). Is pedagogical agent in multimedia learning good for learning? A meta-analysis. Advances in Psychological Science, 25(1), 12. https://doi.org/10.3724/sp.j.1042.2017.00012
[31]
Yoshida, A., Kumazaki, H., Muramatsu, T., Yoshikawa, Y., Ishiguro, H., & Mimura, M. (2022). Intervention with a humanoid robot avatar for individuals with social anxiety disorders comorbid with autism spectrum disorders. Asian Journal of Psychiatry, 78, 103315. https://doi.org/10.1016/j.ajp.2022.103315
Index Terms
- The Potential of Learning With AI-Generated Pedagogical Agents in Instructional Videos
Index terms have been assigned to the content through auto-classification.
Recommendations
Are Pedagogical Agents' External Regulation Effective in Fostering Learning with Intelligent Tutoring Systems?
ITS 2016: Proceedings of the 13th International Conference on Intelligent Tutoring Systems - Volume 9684In this study we tested whether external regulation provided by artificial pedagogical agents PAs was effective in facilitating learners' self-regulated learning SRL and can therefore foster complex learning with a hypermedia-based intelligent tutoring ...
Can Adaptive Pedagogical Agents' Prompting Strategies Improve Students' Learning and Self-Regulation?
ITS 2016: Proceedings of the 13th International Conference on Intelligent Tutoring Systems - Volume 9684This study examines whether an ITS that fosters the use of metacognitive strategies can benefit from variations in its prompts based on learners' self-regulatory behaviors. We use log files and questionnaire data from 116 participants who interacted ...
Increasing learners' motivation through pedagogical agents: the cast of virtual characters in the dynalearn ILE
AEGS'11: Proceedings of the 2011 international conference on Agents for Educational Games and SimulationsMotivation is a critical requirement for successful learning. Previous research has identified that animated pedagogical agents can increase motivation. Following these results, we present the cast of pedagogical agents in the DynaLearn Intelligent ...
Comments
Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI 2025
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 1,519Total Downloads
- Downloads (Last 12 months)1,519
- Downloads (Last 6 weeks)181
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format