research-article

How video production affects student engagement: an empirical study of MOOC videos

Authors:

Rob RubinAuthors Info & Claims

L@S '14: Proceedings of the first ACM conference on Learning @ scale conference

Pages 41 - 50

https://doi.org/10.1145/2556325.2566239

Published: 04 March 2014 Publication History

Abstract

Videos are a widely-used kind of resource for online learning. This paper presents an empirical study of how video production decisions affect student engagement in online educational videos. To our knowledge, ours is the largest-scale study of video engagement to date, using data from 6.9 million video watching sessions across four courses on the edX MOOC platform. We measure engagement by how long students are watching each video, and whether they attempt to answer post-video assessment problems.

Our main findings are that shorter videos are much more engaging, that informal talking-head videos are more engaging, that Khan-style tablet drawings are more engaging, that even high-quality pre-recorded classroom lectures might not make for engaging online videos, and that students engage differently with lecture and tutorial videos.

Based upon these quantitative findings and qualitative insights from interviews with edX staff, we developed a set of recommendations to help instructors and video producers take better advantage of the online video format. Finally, to enable researchers to reproduce and build upon our findings, we have made our anonymized video watching data set and analysis scripts public. To our knowledge, ours is one of the first public data sets on MOOC resource usage.

References

[1]

Khan Academy YouTube Channel. http://www.youtube.com/user/khanacademy/about.

[2]

Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., and Seaton, D. T. Studying learning in the worldwide classroom: Research into edX's first MOOC. Research and Practice in Assessment 8 (Summer 2013).

[3]

Coetzee, D., Fox, A., Hearst, M. A., and Hartmann, B. Should Your MOOC Forum Use a Reputation System? CSCW'14, ACM (New York, NY, USA, 2014).

Digital Library

[4]

Cross, A., Bayyapunedi, M., Cutrell, E., Agarwal, A., and Thies, W. TypeRighting: Combining the Benefits of Handwriting and Typeface in Online Educational Videos. CHI'13, ACM (New York, NY, USA, 2013).

Digital Library

[5]

Google. How Visits are calculated in Analytics. https://support.google.com/analytics/answer/2731565?hl=en.

[6]

Google. YouTube Analytics. http://www.youtube.com/yt/playbook/yt-analytics.html#details.

[7]

Haber, J. xMOOC vs. cMOOC. http://degreeoffreedom.org/xmooc-vs-cmooc/, 2013.

[8]

Ilioudi, C., Giannakos, M. N., and Chorianopoulos, K. Investigating Differences among the Commonly Used Video Lecture Styles. In Proceedings of the Workshop on Analytics on Video-based Learning, WAVe'13 (2013).

[9]

Kizilcec, R. F., Piech, C., and Schneider, E. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, LAK '13, ACM (New York, NY, USA, 2013), 170--179.

Digital Library

[10]

Levasseur, D. G., and Sawyer, J. K. Pedagogy Meets PowerPoint: A Research Review of the Effects of Computer-Generated Slides in the Classroom. Review of Communication 6, 1 (2006), 101--123.

[11]

Mayer, R. E. Multimedia Learning. Cambridge University Press, 2001.

Digital Library

[12]

Roam, D. The Back of the Napkin (Expanded Edition): Solving Problems and Selling Ideas with Pictures. Portfolio Hardcover, 2009.

[13]

Seaton, D. T., Bergner, Y., Chuang, I., Mitros, P., and Pritchard, D. E. Who does what in a massive open online course? Communications of the ACM (2013).

[14]

Wade, A., and Koutoumanou, E. Non-parametric tests: Confidence intervals for a single median. https://epilab.ich.ucl.ac.uk/coursematerial/statistics/non_parametric/confidence_interval.html.

[15]

Williams, J. R. Guidelines for the use of multimedia in instruction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 42, 20 (1998), 1447--1451.

[16]

Wistia. Does length matter? It does for video! http://wistia.com/blog/does-length-matter-it-does-for-video, Sept. 2013.

Cited By

Mohd Khairi MYahya ANik Farid NAga Mohd Jaladin RAhmad Tajuddin N(2024)Development and Validation of a Psychoeducational Video on DepressionCureus10.7759/cureus.59347Online publication date: 30-Apr-2024
https://doi.org/10.7759/cureus.59347
Liu XLi C(2024)The Effects of Social Cues in Self-Produced Micro Video Lectures on Parasocial Interaction, Motivational Interest, Extraneous Cognitive Load, and Concept Learning Performance: an Empirical Study in a Junior High SchoolInternational Journal of Sociologies and Anthropologies Science Reviews10.60027/ijsasr.2024.51354:6(621-638)Online publication date: 11-Nov-2024
https://doi.org/10.60027/ijsasr.2024.5135
Baykov APetrosyan L(2024)ԲԱՐՁՐԱԳՈՒՅՆ ԲԺՇԿԱԿԱՆ ԿՐԹՈՒԹՅԱՆ ՄԵՋ ՈՒՍՈՒՑՄԱՆ «ՇՐՋՎԱԾ ԴԱՍԱՐԱՆ» ՄՈԴԵԼԻ ՆԵՐԴՆՄԱՆ ՀԵՌԱՆԿԱՐՆԵՐԸMEDICINE, SCIENCE AND EDUCATION10.56936/18291775-2024.37-106(106-114)Online publication date: May-2024
https://doi.org/10.56936/18291775-2024.37-106
Show More Cited By

Index Terms

How video production affects student engagement: an empirical study of MOOC videos
1. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Criteria for Video Engagement in a Biology MOOC
L@S '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale

We have designed a Molecular Biology massive open online course (MOOC) for a global audience. Among the learning aids offered are two types of short video segments: lecture videos (delivered unscripted by a professor) and deep dives (fully scripted and ...
Understanding the Effect of In-Video Prompting on Learners and Instructors
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Online instructional videos are ubiquitous, but it is difficult for instructors to gauge learners' experience and their level of comprehension or confusion regarding the lecture video. Moreover, learners watching the videos may become disengaged or fail ...
Blended Learning in Indian Colleges with Massively Empowered Classroom
L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ Scale

Students in the developing world are frequently cited as being among the most important beneficiaries of online education initiatives such as massive open online courses (MOOCs). While some predict that online classrooms will replace physical classrooms,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

L@S '14: Proceedings of the first ACM conference on Learning @ scale conference

March 2014

234 pages

ISBN:9781450326698

DOI:10.1145/2556325

General Chair:
Mehran Sahami
Stanford University, USA
,
Program Chairs:
Armando Fox
University of California, Berkeley, USA
,
Marti A. Hearst
University of California, Berkeley, USA
,
Michelene T.H. Chi
Arizona State University, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

ACM Ed Board: ACM Ed Board

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

L@S 2014

Sponsor:

ACM Ed Board

L@S 2014: First (2014) ACM Conference on Learning @ Scale

March 4 - 5, 2014

Georgia, Atlanta, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1,039
Total Citations
View Citations
16,191
Total Downloads

Downloads (Last 12 months)1,432
Downloads (Last 6 weeks)188

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mohd Khairi MYahya ANik Farid NAga Mohd Jaladin RAhmad Tajuddin N(2024)Development and Validation of a Psychoeducational Video on DepressionCureus10.7759/cureus.59347Online publication date: 30-Apr-2024
https://doi.org/10.7759/cureus.59347
Liu XLi C(2024)The Effects of Social Cues in Self-Produced Micro Video Lectures on Parasocial Interaction, Motivational Interest, Extraneous Cognitive Load, and Concept Learning Performance: an Empirical Study in a Junior High SchoolInternational Journal of Sociologies and Anthropologies Science Reviews10.60027/ijsasr.2024.51354:6(621-638)Online publication date: 11-Nov-2024
https://doi.org/10.60027/ijsasr.2024.5135
Baykov APetrosyan L(2024)ԲԱՐՁՐԱԳՈՒՅՆ ԲԺՇԿԱԿԱՆ ԿՐԹՈՒԹՅԱՆ ՄԵՋ ՈՒՍՈՒՑՄԱՆ «ՇՐՋՎԱԾ ԴԱՍԱՐԱՆ» ՄՈԴԵԼԻ ՆԵՐԴՆՄԱՆ ՀԵՌԱՆԿԱՐՆԵՐԸMEDICINE, SCIENCE AND EDUCATION10.56936/18291775-2024.37-106(106-114)Online publication date: May-2024
https://doi.org/10.56936/18291775-2024.37-106
Martinez SHarrington SAchilles CMohammed HTjan P(2024)APPLYING GIBBONS' ARCHITECTURAL INSTRUCTIONAL DESIGN APPROACH TO EVALUATE NON-PROFIT TRAININGPerformance Improvement Journal10.56811/PFI-23-000362:6(205-212)Online publication date: 18-Jul-2024
https://doi.org/10.56811/PFI-23-0003
Al-Qudah E(2024)The Effectiveness of Recorded Lectures on University Students’ AchievementJournal of Law and Sustainable Development10.55908/sdgs.v12i1.253712:1(e2537)Online publication date: 22-Jan-2024
https://doi.org/10.55908/sdgs.v12i1.2537
Meziane Cherif KAzzouz LBendania A(2024)Algerian secondary school students' preferences for the use of YouTube in their informal learningEducational Technology Quarterly10.55056/etq.6972024:2(120-134)Online publication date: 20-Jun-2024
https://doi.org/10.55056/etq.697
Adarsh Bhavimane Rakshitha Shetty Ghrutha Varsha Kurunji Alex Tayenjam Dr. Pushparani M K (2024)Data Visualization in Education: A Comprehensive ReviewInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-18676(503-509)Online publication date: 31-May-2024
https://doi.org/10.48175/IJARSCT-18676
Wu SChen YSoong CChiang CWang WHung C(2024)Flipping the classroom: A comparative analysis of in-person versus online learning in scabies education for medical studentsDermatologica Sinica10.4103/ds.DS-D-24-0018442:4(273-279)Online publication date: 27-Nov-2024
https://doi.org/10.4103/ds.DS-D-24-00184
Mitani H(2024)Understanding the Online Aspect of Educational Leadership ProgramsReimagining the P-20 Landscape for School Leadership Learning10.4018/979-8-3693-6220-4.ch010(237-274)Online publication date: 27-Dec-2024
https://doi.org/10.4018/979-8-3693-6220-4.ch010
Kayabas I(2024)Innovative Curriculum Development and Content Creation With Generative AITransforming Education With Generative AI10.4018/979-8-3693-1351-0.ch022(462-473)Online publication date: 29-Mar-2024
https://doi.org/10.4018/979-8-3693-1351-0.ch022
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents