Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

The influence of learning analytics dashboard information design on cognitive load and performance

Published: 01 April 2024 Publication History

Abstract

Learning analytics dashboards are becoming increasingly common tools for providing feedback to learners. However, there is limited empirical evidence regarding the effects of learning analytics dashboard design features on learners’ cognitive load, particularly in digital learning environments. To address this gap, we developed goal-based, explanatory, and instructional learning analytics dashboards in authentic online courses based on cognitive load theory, and evaluated the effects of the three information designs on cognitive load and performance. The study adopted a quasi-experimental approach over a semester-long course, involving 93 learners divided into four groups, each provided with differently designed information on their learning analytics dashboard. The results show that the incorporation of goals, explanations, and instructional information as support elements in the learning analytics dashboard did not have a significant impact on learners’ cognitive load and performance. Both cognitive load and learning performance results were consistent and mutually validating. Additionally, the study found that compared to a control group without additional information, the group using the explanatory dashboard experienced an increase in germane cognitive load, and evidenced the effectiveness of explanatory information design. Overall, this study provides important insights for the enhancement and practical design of learning analytics dashboards and feedback methods.

References

[1]
Afzaal M, Nouri J, Zia A, Papapetrou P, Fors U, Wu Y, Li X, and Weegar R Explainable AI for Data-Driven Feedback and Intelligent Action recommendations to Support Students Self-Regulation Frontiers in Artificial Intelligence 2021 4 723447
[2]
Aguilar, S. J. (2022). Experimental Evidence of Performance Feedback vs. Mastery Feedback on Students’ Academic Motivation. LAK22: 12th International Learning Analytics and Knowledge Conference, 556–562.
[3]
Albus P, Vogt A, and Seufert T Signaling in virtual reality influences learning outcome and cognitive load Computers & Education 2021 166 104154
[4]
Alhadad SSJ Visualizing Data to Support Judgement 2018
[5]
Antonio Perez-Alvarez R, Maldonado-Mahauad J, Sharma K, Sapunar-Opazo D, and Perez-Sanagustin M Characterizing Learners’ Engagement in MOOCs: An observational case study using the NoteMyProgress Tool for supporting self-regulation IEEE Transactions on Learning Technologies 2020 13 4 676-688
[6]
Bahari A, Wu S, and Ayres P Improving computer-assisted Language Learning through the Lens of Cognitive load Educational Psychology Review 2023 35 2 53
[7]
Barrera-Leon L, Corno F, and De Russis L How the Preattentive process is exploited in practical information visualization design: A review International Journal of Human–Computer Interaction 2023 39 4 707-720
[8]
Becker S, Klein P, Gößling A, and Kuhn J Investigating dynamic visualizations of multiple representations using Mobile Video Analysis in Physics lessons Zeitschrift Für Didaktik Der Naturwissenschaften 2020 26 1 123-142
[9]
Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, and Munafò MR Power failure: Why small sample size undermines the reliability of neuroscience Nature Reviews Neuroscience 2013 14 5 5
[10]
Cha, H. J., & Park, T. (2019). Applying and evaluating visualization design guidelines for a MOOC dashboard to Facilitate Self-regulated learning based on learning analytics. KSII Transactions on Internet and Information Systems, 13(6).
[11]
Chen HJ Clarifying the impact of surprise in e-learning system design based on university students with multiple learning goals orientation Education and Information Technologies 2020 25 6 5873-5892
[12]
Chen CH and Su CY Using the BookRoll E-Book System to Promote Self-regulated learning, self-efficacy and academic achievement for University students Educational Technology & Society 2019 22 4 33-46
[13]
Chen, L., Lu, M., Goda, Y., Shimada, A., & Yamada, M. (2021). Learning analytics Dashboard supporting Metacognition. Balancing the Tension between Digital Technologies and Learning Sciences, 129–149.
[14]
Chen X, Zou D, and Xie H A decade of learning analytics: Structural topic modeling based bibliometric analysis Education and Information Technologies 2022 27 8 10517-10561
[15]
Chevalier M, Giang C, El-Hamamsy L, Bonnet E, Papaspyros V, Pellet JP, Audrin C, Romero M, Baumberger B, and Mondada F The role of feedback and guidance as intervention methods to foster computational thinking in educational robotics learning activities for primary school Computers & Education 2022 180 104431
[16]
Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic press.
[17]
Corrin, L., & Barba, P. D. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education 2014.
[18]
Costley J Using cognitive strategies overcomes cognitive load in online learning environments Interactive Technology and Smart Education 2020 17 2 215-228
[19]
Deeva G, Bogdanova D, Serral E, Snoeck M, and De Weerdt J A review of automated feedback systems for learners: Classification framework, challenges and opportunities Computers & Education 2021 162 104094
[20]
Duran R, Zavgorodniaia A, and Sorva J Cognitive load theory in Computing Education Research: A review ACM Transactions on Computing Education 2022 22 4 1-27
[21]
Dyckhoff, A., Zielke, D., Bültmann, M., Chatti, M., & Schroeder, U. (2012). Design and implementation of a Learning Analytics Toolkit for teachers. Educational Technology and Society, 15(3), 58-76.
[22]
Glogger-Frey I, Gaus K, and Renkl A Learning from direct instruction: Best prepared by several self-regulated or guided invention activities? Learning and Instruction 2017 51 26-35
[23]
Greller W and Drachsler H Translating learning into numbers: A generic framework for learning analytics Journal of Educational Technology & Society 2012 15 3 42-57
[24]
Hauer KE, Iverson N, Quach A, Yuan P, Kaner S, and Boscardin C Fostering medical students’ lifelong learning skills with a dashboard, coaching and learning planning Perspectives on Medical Education 2018 7 5 311-317
[25]
Hedges, L. V. (2008). What are effect sizes and why do we need them? Child Development perspectives, 2(3), 167–171.
[26]
Huang, L., Zheng, J., Lajoie, S. P., Chen, Y., Hmelo-Silver, C. E., & Wang, M. (2023). Examining university teachers’ self-regulation in using a learning analytics dashboard for online collaboration. Education and Information Technologies, 1–25.
[27]
Jan SL and Shieh G Sample size determinations for Welch’s test in one-way heteroscedastic ANOVA British Journal of Mathematical and Statistical Psychology 2014 67 1 72-93
[28]
Jivet I, Scheffel M, Schmitz M, Robbers S, Specht M, and Drachsler H From students with love: An empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education The Internet and Higher Education 2020 47 100758
[29]
Jovanović J, Gašević D, Pardo A, Dawson S, and Whitelock-Wainwright A Introducing meaning to clicks: Towards Traced-measures of Self-Efficacy and cognitive load Proceedings of the 9th International Conference on Learning Analytics & Knowledge 2019 511 520
[30]
Kia, F. S., Teasley, S. D., Hatala, M., Karabenick, S. A., & Kay, M. (2020). How Patterns of Students Dashboard Use Are Related to Their Achievement and Self-Regulatory Engagement. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 340–349.
[31]
Kopp T, Riekert M, and Utz S When cognitive fit outweighs cognitive load: Redundant data labels in charts increase accuracy and speed of information extraction Computers in Human Behavior 2018 86 367-376
[32]
Krieglstein F, Beege M, Rey GD, Ginns P, Krell M, and Schneider S A systematic Meta-analysis of the reliability and validity of subjective cognitive load questionnaires in experimental Multimedia Learning Research Educational Psychology Review 2022
[33]
Leppink J, Paas F, Van der Vleuten CPM, Van Gog T, and Van Merriënboer JJG Development of an instrument for measuring different types of cognitive load Behavior Research Methods 2013 45 4 1058-1072
[34]
Li Q, Xu D, Baker R, Holton A, and Warschauer M Can student-facing analytics improve online students’ effort and success by affecting how they explain the cause of past performance? Computers & Education 2022 185 104517
[35]
Li W, Feng Q, Zhu X, Yu Q, and Wang Q Effect of summarizing scaffolding and textual cues on learning performance, mental model, and cognitive load in a virtual reality environment: An experimental study Computers & Education 2023 200 104793
[36]
Liao CW, Chen CH, and Shih SJ The interactivity of video and collaboration for learning achievement, intrinsic motivation, cognitive load, and behavior patterns in a digital game-based learning environment Computers & Education 2019 133 43-55
[37]
Locke EA and Latham GP Building a practically useful theory of goal setting and task motivation: A 35-year odyssey American Psychologist 2002 57 9 705-717
[38]
Manganello F, Pozzi F, Passarelli M, Persico D, and Dagnino FM A dashboard to monitor self-regulated Learning behaviours in Online Professional Development International Journal of Distance Education Technologies 2021 19 1 18-34
[39]
Mao P, Cai Z, Wang Z, Hao X, Fan X, and Sun X The effects of dynamic and static feedback under tasks with different difficulty levels in digital game-based learning The Internet and Higher Education 2024 60 100923
[40]
Matcha W, Uzir NA, Gasevic D, and Pardo A A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective IEEE Transactions on Learning Technologies 2020 13 2 226-245
[41]
Mayer, R. E., & Fiorella, L. (Eds.). (2021). The Cambridge Handbook of Multimedia Learning (3rd ed.). Cambridge University Press.
[42]
Moraes Bueno Rodrigues A, Diniz Junqueira Barbosa G, Côrtes Vieira Lopes H, Diniz J, and Barbosa S What questions reveal about novices’ attempts to make sense of data visualizations: Patterns and misconceptions Computers & Graphics 2021 94 32-42
[43]
Mutlu-Bayraktar D, Cosgun V, and Altan T Cognitive load in multimedia learning environments: A systematic review Computers & Education 2019 141 103618
[44]
Nebel S, Schneider S, Schledjewski J, and Rey GD Goal-setting in Educational Video games: Comparing goal-setting theory and the goal-free effect Simulation & Gaming 2017 48 1 98-130
[45]
Park Y and Jo IH Factors that affect the success of learning analytics dashboards Educational Technology Research and Development 2019 67 6 1547-1571
[46]
Paulsen L and Lindsay E Learning analytics dashboards are increasingly becoming about learning and not just analytics—A systematic review Education and Information Technologies 2024
[47]
Pranoto YKS Achievement goals and extraneous load predict germane load: The mediating effects of achievement emotions Malaysian Journal of Learning and Instruction (MJLI) 2021 18 2 215-234
[48]
Raaijmakers SF, Baars M, Schaap L, Paas F, and van Gog T Effects of performance feedback valence on perceptions of invested mental effort Learning and Instruction 2017 51 36-46
[49]
Ramaswami G, Susnjak T, Mathrani A, and Umer R Use of Predictive analytics within Learning Analytics dashboards: A review of Case studies Technology Knowledge and Learning 2023 28 3 959-980
[50]
Redifer JL, Bae CL, and DeBusk-Lane M Implicit theories, working memory, and cognitive load: Impacts on creative thinking SAGE Open 2019 9 1 2158244019835919
[51]
Redifer JL, Bae CL, and Zhao Q Self-efficacy and performance feedback: Impacts on cognitive load during creative thinking Learning and Instruction 2021 71 101395
[52]
Rets I, Herodotou C, Bayer V, Hlosta M, and Rienties B Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students International Journal of Educational Technology in Higher Education 2021 18 1 46
[53]
Şahin M and Yurdugül H An intervention engine design and development based on learning analytics: The intelligent intervention system (In2S) Smart Learning Environments 2019 6 1 18
[54]
Schroeder NL and Cenkci AT Spatial contiguity and spatial Split-attention effects in Multimedia Learning environments: A Meta-analysis Educational Psychology Review 2018 30 3 679-701
[55]
Schumacher C and Ifenthaler D Features students really expect from learning analytics Computers in Human Behavior 2018 78 397-407
[56]
Schwendimann BA, Rodríguez-Triana MJ, Vozniuk A, Prieto LP, Boroujeni MS, Holzer A, Gillet D, and Dillenbourg P Perceiving learning at a glance: A systematic literature review of learning Dashboard Research IEEE Transactions on Learning Technologies 2017 10 1 30-41
[57]
Sedrakyan, G., Järvelä, S., & Kirschner, P. (2016). Conceptual framework for feedback automation and personalization for designing learning analytics dashboards. In EARLI SIG 2016, Date: 2016/11/30-2016/12/01, Location: Oulu, Finland.
[58]
Sedrakyan, G., Mannens, E., & Verbert, K. (2019). Guiding the choice of learning dashboard visualizations: Linking dashboard design and data visualization concepts. Journal of Computer Languages, 50, 19–38.
[59]
Seufert T The interplay between self-regulation in learning and cognitive load Educational Research Review 2018 24 116-129
[60]
Skulmowski A Guidelines for choosing cognitive load measures in Perceptually Rich environments Mind Brain and Education 2023 17 1 20-28
[61]
Skulmowski A and Xu KM Understanding cognitive load in Digital and Online Learning: A new perspective on extraneous cognitive load Educational Psychology Review 2022 34 1 171-196
[62]
Srivastava, N., Nawaz, S., Lodge, J. M., Velloso, E., Erfani, S., & Bailey, J. (2020). Exploring the usage of thermal imaging for understanding video lecture designs and students’ experiences. In Proceedings of the tenth international conference on learning analytics & knowledge, 250–259.
[63]
Sun, W., Schumacher, C., Chen, L., & Pinkwart, N. (2021). What do MOOC dashboards present to Learners? Visualizations and Dashboards for Learning Analytics, 117–148.
[64]
Swart EK, Nielen TMJ, and de Sikkema MT Supporting learning from text: A meta-analysis on the timing and content of effective feedback Educational Research Review 2019 28 100296
[65]
Sweller J Element interactivity and intrinsic, extraneous, and germane cognitive load Educational Psychology Review 2010 22 2 123-138
[66]
Sweller, J. (2011). Cognitive load theory. In Psychology of learning and motivation (Vol. 55, pp. 37–76). Academic Press.
[67]
Sweller J Cognitive load theory and educational technology Educational Technology Research and Development 2020 68 1 1-16
[68]
Sweller J, van Merriënboer JJG, and Paas F Cognitive Architecture and Instructional Design: 20 years later Educational Psychology Review 2019 31 2 261-292
[69]
Valle N, Antonenko P, Valle D, Dawson K, Huggins-Manley AC, and Baiser B The influence of task-value scaffolding in a predictive learning analytics dashboard on learners’ statistics anxiety, motivation, and performance Computers & Education 2021 173 104288
[70]
Van der Kleij FM, Feskens RCW, and Eggen TJHM Effects of Feedback in a computer-based learning environment on students’ learning outcomes: A Meta-analysis Review of Educational Research 2015 85 4 475-511
[71]
Van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL: Balancing between overview and overload. Journal of Learning Analytics, 2(2).
[72]
Vanneste P, Raes A, Morton J, Bombeke K, Van Acker BB, Larmuseau C, Depaepe F, and Van den Noortgate W Towards measuring cognitive load through multimodal physiological data Cognition Technology & Work 2021 23 3 567-585
[73]
Vieira C, Parsons P, and Byrd V Visual learning analytics of educational data: A systematic literature review and research agenda Computers & Education 2018 122 119-135
[74]
Wang Z, Gong SY, Xu S, and Hu XE Elaborated feedback and learning: Examining cognitive and motivational influences Computers & Education 2019 136 130-140
[75]
Wang T, Li S, Tan C, Zhang J, and Lajoie SP Cognitive load patterns affect temporal dynamics of self-regulated learning behaviors, metacognitive judgments, and learning achievements Computers & Education 2023 207 104924
[76]
Weber KE, Prilop CN, and Kleinknecht M Effects of different video- or text-based reflection stimuli on pre-service teachers’ emotions, immersion, cognitive load and knowledge-based reasoning Studies in Educational Evaluation 2023 77 101256
[77]
Wiedbusch, M. D., Kite, V., Yang, X., Park, S., Chi, M., Taub, M., & Azevedo, R. (2021). A Theoretical and Evidence-Based Conceptual Design of MetaDash: An Intelligent Teacher Dashboard to Support Teachers’ Decision Making and Students’ Self-Regulated Learning. Frontiers in Education, 6. https://www.frontiersin.org/article/.
[78]
Wirth J, Künsting J, and Leutner D The impact of goal specificity and goal type on learning outcome and cognitive load Computers in Human Behavior 2009 25 2 299-305
[79]
Xu KM, Koorn P, de Koning B, Skuballa IT, Lin L, Henderikx M, Marsh HW, Sweller J, and Paas F A growth mindset lowers perceived cognitive load and improves learning: Integrating motivation to cognitive load Journal of Educational Psychology 2021 113 6 1177-1191
[80]
Yang CCY and Ogata H Personalized learning analytics intervention approach for enhancing student learning achievement and behavioral engagement in blended learning Education and Information Technologies 2023 28 3 2509-2528
[81]
Yoon M, Hill J, and Kim D Designing supports for promoting self-regulated learning in the flipped classroom Journal of Computing in Higher Education 2021 33 2 398-418
[82]
Zheng L, Zhong L, and Niu J Effects of personalised feedback approach on knowledge building, emotions, co-regulated behavioural patterns and cognitive load in online collaborative learning Assessment & Evaluation in Higher Education 2022 47 1 109-125
[83]
Zheng L, Long M, Niu J, and Zhong L An automated group learning engagement analysis and feedback approach to promoting collaborative knowledge building, group performance, and socially shared regulation in CSCL International Journal of Computer-Supported Collaborative Learning 2023 18 1 101-133
[84]
Zhu M Enhancing MOOC learners’ skills for self-directed learning Distance Education 2021 42 3 441-460

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Education and Information Technologies
Education and Information Technologies  Volume 29, Issue 15
Oct 2024
1537 pages

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 April 2024
Accepted: 28 February 2024
Received: 03 May 2023

Author Tags

  1. Learning analytics dashboard
  2. Information design
  3. Cognitive load
  4. Learning performance

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 11 Feb 2025

Other Metrics

Citations

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media