article

Evaluating learning, design, and engagement in web-based learning tools (WBLTs): The WBLT Evaluation Scale

Author:

Robin KayAuthors Info & Claims

Computers in Human Behavior, Volume 27, Issue 5

Pages 1849 - 1856

https://doi.org/10.1016/j.chb.2011.04.007

Published: 01 September 2011 Publication History

Abstract

Web-based learning tools (WBLTs), also known as learning objects, are online, interactive tools that support the learning of specific concepts by enhancing, amplifying, and/or guiding the cognitive processes of learners. Research examining the effectiveness of WBLTs is somewhat limited because sound, reliable, valid evaluation metrics are sparse, particularly in the K-12 environment. The purpose of the following study was to re-examine the Learning Object Evaluation Scale for students (LOES-S), originally developed by Kay and Knaack (2009), to assess three key constructs: learning, design, and engagement. Over 800 middle and secondary schools students participated in high quality, pre-designed lessons intended to accentuate the use of WBLTs. Data collected from the new WBLT Evaluation Scale demonstrated good internal reliability, construct validity, convergent validity and predictive validity.

References

[1]

A coaching agent for learners using multimedia simulations. Educational Technology. v37 i2. 44-49.

[2]

. In: Anderson, L.W., Krathwohl, D.R. (Eds.), A taxonomy for learning, teaching, and assessing: A revision of bloom's taxonomy of educational objectives, Longman, New York.

[3]

Transformative power of the learning object debate. Open Learning. v20 i1. 15-30.

[4]

The design, development, and use of multimedia learning objects. Journal of Educational Multimedia and Hypermedia. v13 i4. 371-389.

[5]

Elements of effective e-learning design. The International Review of Research in Open and Distance Learning. v6 i1.

[6]

The use of embedded scaffolds with hypermedia-supported student-centered learning. Journal of Educational Multimedia and Hypermedia. v10 i4. 333-356.

[7]

Guidelines and standards for the development of fully online learning objects. Interdisciplinary Journal of Knowledge and Learning Objects. v2. 96-104.

[8]

Navigation and disorientation: A case study. Journal of Educational Multimedia and Hypermedia. v6 i3/4. 305-320.

Digital Library

[9]

Clarke, O., & Bowe, L. (2006a). The learning federation and the Victorian department of education and training trial of online curriculum content with Indigenous students (pp. 1-14). <http://www.sofweb.vic.edu.au/edulibrary/public/teachlearn/ict/TLF_DETVIC_indig_trial_mar06.pdf>.

[10]

Clarke, O., & Bowe, L. (2006b). The learning federation and the Victorian department of education and training trial of online curriculum content with ESL students (pp. 1-16). <http://www.thelearningfederation.edu.au/verve/_resources/report_esl_final.pdf>.

[11]

Interactive QuickTime: Developing and evaluating multimedia learning objects to enhance both face-to-face and distance e-learning environments. Interdisciplinary Journal of Knowledge and Learning Objects. v1. 33-54.

[12]

Del Moral, E. & Cernea, D. A. (2005). Design and evaluate learning objects in the new framework of the semantic web. In A. Mendez-Vila, B. Gonzalez-Pereira, J. Mesa Gonzalez, & J.A. Mesa Gonsalez (Eds.), Recent research developments in learning technologies (pp. 1-5). <http://www.formatex.org/micte2005/357.pdf>.

[13]

E-learning techniques supporting problem based learning in clinical simulation. International Journal of Medical Informatics. v74 i7-8. 527-533.

[14]

Discovering statistics using SPSS. 2nd ed. SAGE Publications, Thousand Oaks, CA.

Digital Library

[15]

Designing online mathematical investigation. Journal of Computers in Mathematics and Science Teaching. v23 i3. 275-298.

[16]

Relation of sample size to the stability of component patterns. Psychological Bulletin. v103 i2. 265-275.

[17]

Evaluating learning objects for schools. Australasian Journal of Educational Technology. v21 i4. 470-490.

[18]

Howard-Rose, D., & Harrigan, K. (2003) CLOE learning impact studies lite: Evaluating learning objects in nine Ontario University courses. <http://cloe.on.ca/documents/merlotconference10.doc>.

[19]

Is there a learning orientation in learning objects?. International Journal on E-Learning. v3 i2. 32-41.

[20]

Using metacognitive theories to design user interfaces for computer-based learning. Educational Technology. v35 i4. 12-22.

[21]

Kay, R. H. (2011). Appendix A - List of WBLTs used in secondary school study. <http://tiny.cc/wblt_appendix_a>.

[22]

Kay, R. H., Knaack, L., & Petrarca, D. (2009). Exploring teacher perceptions of Web-based learning tools. 5, 27-50. <http://www.ijello.org/Volume5/IJELLOv5p027-050Kay649.pdf>.

[23]

Developing learning objects for secondary school students: A multi-component model. Interdisciplinary Journal of E-Learning and Learning Objects. v1. 229-254.

[24]

Evaluating the learning in learning objects. Open Learning. v22 i1. 5-28.

[25]

A multi-component model for assessing learning objects: The learning object evaluation metric (LOEM). Australasian Journal of Educational Technology. v24 i5. 574-591.

[26]

Assessing learning, quality and engagement in learning objects: The learning object evaluation scale for students (LOES-S). Education Technology Research and Development. v57 i2. 147-168.

[27]

A formative analysis of instructional strategies for using learning objects. Journal of Interactive Learning Research. v20 i3. 295-315.

[28]

Design elements for interactive multimedia. Australian Journal of Educational Technology. v13 i1. 1-22.

[29]

Towards guidelines for the design of interactive multimedia instruction: Fostering the reflective decision-making of preservice teachers. Journal of Technology and Teacher Education. v7 i1. 13-31.

Digital Library

[30]

The handbook of psychological testing. 2nd ed. Routledge, London.

[31]

Architecting usability properties in the e-learning instructional design process. International Journal on E-Learning. v3 i3. 38-44.

[32]

Learning objects, repositories, sharing and reusability. Open Learning. v20 i1. 83-91.

[33]

Using technology to enhance mathematical reasoning: Effects of feedback and self-regulation learning. Education Media International. v38 i2/3. 77-82.

[34]

A study of the design and evaluation of a learning object and implications for content development. Interdisciplinary Journal of Knowledge and Learning Objects. v1. 1-22.

[35]

Developing interactive learning objects for a computing mathematics models. International Journal on E-Learning. v5 i2. 221-244.

[36]

Designing websites for learning and enjoyment: A study of museum experiences. International Review of Research in Open and Distance Learning. v7 i3. 1-21.

[37]

An analysis of cognitive tool use patterns in a hypermedia learning environment. Educational Technology, Research and Development. v53 i1. 5-21.

[38]

Computer support for learning mathematics: A learning environment based on recreational learning objects. Computers and Education. v48 i4. 618-641.

Digital Library

[39]

Addressing the eLearning contradiction: A collaborative approach for developing a conceptual framework learning object. Interdisciplinary Journal of Knowledge and Learning Objects. v1. 79-98.

[40]

Twenty-one guidelines for effective instructional design. Educational Technology. v35 i3. 58-61.

[41]

Aids to computer-based multimedia learning. Learning and Instruction. v12 i1. 107-119.

[42]

Learning objects: A practical definition. International Journal of Instructional Technology and Distance Learning. v1 i9.

[43]

EduSource: Canada's learning object repository network. International Journal of Instructional Technology and Distance Learning. v1 i3.

[44]

Collaborative evaluation of learning objects. In: McGreal, R. (Ed.), Online education using learning objects, RoutledgeFalmer, New York. pp. 138-153.

[45]

Nielson, J. (2003). Ten usability heuristics. <http://www.useit.com/papers/heuristic/heuristic_list.html>.

[46]

Psychometric theory. McGraw-Hill, New York.

[47]

Problems underlying the learning object approach. International Journal of Instructional Technology and Distance Learning. v2 i11.

[48]

Effectiveness of learning objects in various instructional settings. Learning, Media, and Technology. v31 i3. 233-247.

[49]

Promises and pitfall of learning objects. Learning, Media, and Technology. v31 i3. 269-285.

[50]

An interaction-centric learning model. Journal of Educational Multimedia and Hypermedia. v10 i4. 311-332.

[51]

Curriculum and learning-resources issues arising from the use of web-based course support systems. International Journal of Educational Telecommunications. v5 i4. 419-435.

[52]

The trouble with learning objects. Educational Technology Research and Development. v52 i1. 49-67.

[53]

Third-graders learning about fractions using virtual manipulatives: A classroom study. Journal of Computers in Mathematics and Science Teaching. v24 i1. 5-25.

[54]

A repository of e-learning objects for higher education. E-Service Journal. v1 i2. 53-64.

[55]

Learning objects in use: 'Lite' assessment for field studies. Journal of Online Learning and Teaching. v1 i1. 1-18.

[56]

Increasing the semantics of learning objects. International Journal of Computer Processing of Oriental Languages. v17 i1. 27-39.

[57]

When is a learning object not an object: A first step towards a theory of learning objects. International Review of Research in Open and Distance Learning. v3 i2. 1-16.

[58]

Applied multivariate statistics for the social science applications. 2nd ed. Erlbaum, Hillsdale, NJ.

[59]

The use of COCOON in teaching silviculture. Computers and Education. v47 i3. 245-259.

Digital Library

[60]

Research on cognitive load theory and its design implications for e-learning. Education Theory, Research and Development. v53 i3. 1042-1629.

[61]

Implementation and evaluation of a course concept based on reusable learning objects. Journal of Educational Computing and Research. v28 i4. 355-372.

[62]

Learning object evaluation: Computer mediated collaboration and inter-rater reliability. International Journal of Computers and Applications. v25 i3. 1-8.

[63]

Overcoming the limitations of learning objects. Journal of Educational Multimedia and Hypermedia. v13 i4. 507-521.

[64]

Williams, D. D. (2000). Evaluation of learning objects and instruction using learning objects. In D. A. Wiley (Ed.), The instructional use of learning objects. <http://reusability.org/read/chapters/williams.doc>.

Cited By

Campos THounsell MDamasceno EValentim N(2024)Evaluating Usability, User Experience, and Playability of a Puzzle Game in Mixed RealityProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691584(61-70)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3691573.3691584
Dhiman HRöcker C(2024)Does the Medium Matter? A Comparison of Augmented Reality Media in Instructing Novices to Perform Complex, Skill-Based Manual Tasks.Proceedings of the ACM on Human-Computer Interaction10.1145/36602498:EICS(1-28)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3660249
Holly MHabich LPirker J(2024)GameDevDojo - An Educational Game for Teaching Game Development ConceptsProceedings of the 19th International Conference on the Foundations of Digital Games10.1145/3649921.3649992(1-9)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3649921.3649992
Show More Cited By

Recommendations

Towards a Conceptual Framework of GBL Design for Engagement and Learning of Curriculum-based Content

This paper aims to show best practices of GBL design for engagement. It intends to show how teachers can implement GBL in a collaborative, comprehensive and systematic way, in the classrooms, and probably outside the classrooms, based on empirical ...
Strategies for Enhancing and Evaluating Interactivity in Web-Based Learning and Teaching

Interactivity is often discussed in constructivist approaches to education, which enable the student to evolve, understand, or construct new ideas from existing concepts. Constructivist approaches rely on active participation in the learning situation. ...
Promoting self-regulated learning in web-based learning environments

Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and ...

Comments

Information & Contributors

Information

Published In

cover image Computers in Human Behavior

Computers in Human Behavior Volume 27, Issue 5

September, 2011

591 pages

ISSN:0747-5632

Issue’s Table of Contents

Copyright © Elsevier Ltd © 2011.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 September 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Campos THounsell MDamasceno EValentim N(2024)Evaluating Usability, User Experience, and Playability of a Puzzle Game in Mixed RealityProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691584(61-70)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3691573.3691584
Dhiman HRöcker C(2024)Does the Medium Matter? A Comparison of Augmented Reality Media in Instructing Novices to Perform Complex, Skill-Based Manual Tasks.Proceedings of the ACM on Human-Computer Interaction10.1145/36602498:EICS(1-28)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3660249
Holly MHabich LPirker J(2024)GameDevDojo - An Educational Game for Teaching Game Development ConceptsProceedings of the 19th International Conference on the Foundations of Digital Games10.1145/3649921.3649992(1-9)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3649921.3649992
Holly MBrettschuh STiwari ABhagat KPirker J(2024)Game-Based Motivation: Enhancing Learning with Achievements in a Customizable Virtual Reality EnvironmentProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687741(1-11)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3641825.3687741
Liu CTan FZhao SKanneganti ATushar GKhoo E(2024)Facilitating Virtual Reality Integration in Medical Education: A Case Study of Acceptability and Learning Impact in Childbirth Delivery TrainingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642100(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642100
Boel CRotsaert TValcke MVanhulsel ASchellens T(2024)Applying educational design research to develop a low-cost, mobile immersive virtual reality serious game teaching safety in secondary vocational educationEducation and Information Technologies10.1007/s10639-023-12126-429:7(8609-8646)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s10639-023-12126-4
Cheng ARitchie JAgrawal NChilds EDeVeaux CJee YLeon TMaples BCuadra ALanday J(2023)Designing Immersive, Narrative-Based Interfaces to Guide Outdoor LearningProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581365(1-22)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581365
Liu YLin YShi RLuo YLiang H(2021)RelicVR: A Virtual Reality Game for Active Exploration of Archaeological RelicsExtended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play10.1145/3450337.3483507(326-332)Online publication date: 15-Oct-2021
https://dl.acm.org/doi/10.1145/3450337.3483507
Veneruso SFerro LMarrella AMecella MCatarci T(2020)CyberVRProceedings of the 2020 International Conference on Advanced Visual Interfaces10.1145/3399715.3399860(1-8)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3399715.3399860
Morfidi EMikropoulos ARogdaki A(2018)Using concept mapping to improve poor readers' understanding of expository textEducation and Information Technologies10.1007/s10639-017-9600-723:1(271-286)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1007/s10639-017-9600-7
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents