research-article

Refocusing the lens on engagement in MOOCs

Authors:

Michelle Perry,

Lawrence Angrave,

Najmuddin Shaik,

Suma BhatAuthors Info & Claims

L@S '18: Proceedings of the Fifth Annual ACM Conference on Learning at Scale

Article No.: 11, Pages 1 - 10

https://doi.org/10.1145/3231644.3231658

Published: 26 June 2018 Publication History

Abstract

Massive open online courses (MOOCs) continue to see increasing enrollment and adoption by universities, although they are still not fully understood and could perhaps be significantly improved. For example, little is known about the relationships between the ways in which students choose to use MOOCs (e.g., sampling lecture videos, discussing topics with fellow students) and their overall level of engagement with the course, although these relationships are likely key to effective course implementation. In this paper we propose a multilevel definition of student engagement with MOOCs and explore the connections between engagement and students' behaviors across five unique courses. We modeled engagement using ordinal penalized logistic regression with the least absolute shrinkage and selection operator (LASSO), and found several predictors of engagement that were consistent across courses. In particular, we found that discussion activities (e.g., viewing forum posts) were positively related to engagement, whereas other types of student behaviors (e.g., attempting quizzes) were consistently related to less engagement with the course. Finally, we discuss implications of unexpected findings that replicated across courses, future work to explore these implications, and relevance of our findings for MOOC course design.

References

[1]

Alan Agresti. 2002. Categorical Data Analysis (2nd. ed.). Wiley Interscience.

[2]

Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web. ACM, 687--698.

Digital Library

[3]

Juan Miguel L. Andres, Ryan S. Baker, George Siemens, Dragen Gašević, and Catherine A. Spann. 2017. Replicating 21 findings on student success in online learning. Technology, Instruction, Cognition, and Learning 10, 4 (2017), 313--333.

[4]

PG de Barba, Gregor E Kennedy, and MD Ainley. 2016. The role of students' motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning 32, 3 (2016), 218--231.

Digital Library

[5]

Christopher Brooks, Craig Thompson, and Stephanie Teasley. 2015. Who you are or what you do: Comparing the predictive power of demographics vs. activity patterns in massive open online courses (MOOCs). In Proceedings of the Second (2015) ACM Conference on Learning@Scale. ACM, 245--248.

Digital Library

[6]

Cho Kin Cheng, Dwayne E. Paré, Lisa-Marie Collimore, and Steve Joordens. 2011. Assessing the effectiveness of a voluntary online discussion forum on improving students' course performance. Computers & Education 56, 1 (2011), 253--261.

Digital Library

[7]

Doug Clow. 2013. MOOCs and the funnel of participation. In Proceedings of the Third International Conference on Learning Analytics and Knowledge. ACM, 185--189.

Digital Library

[8]

Scott Crossley, Mihai Dascalu, Danielle S. McNamara, Ryan S. Baker, and Stefan Trausan-Matu. 2017. Predicting success in massive open online courses (MOOCs) using cohesion network analysis. In Making a Difference: Prioritizing Equity and Access in CSCL, 12th International Conference on Computer Supported Collaborative Learning (CSCL) 2017, Volume 1. Philadelphia, PA: International Society of the Learning Sciences, 103--110.

[9]

Scott Crossley, Luc Paquette, Mihai Dascalu, Danielle S. McNamara, and Ryan S. Baker. 2016. Combining click-stream data with NLP tools to better understand MOOC completion. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. ACM, 6--14.

Digital Library

[10]

R. Wes Crues, Genevieve M. Henricks, Michelle Perry, Suma Bhat, Carolyn J. Anderson, Najmuddin Shaik, and Lawrence Angrave. 2018. How do gender, learning goals, and forum participation predict persistence in a computer science MOOC? ACM Transactions on Computing Education (2018).

[11]

Jennifer DeBoer, Andrew D. Ho, Glenda S. Stump, and Lori Breslow. 2014. Changing "course": Reconceptualizing educational variables for massive open online courses. Educational Researcher 43, 2 (2014), 74--84.

[12]

Lu Ding, ChanMin Kim, and Michael Orey. 2017. Studies of student engagement in gamified online discussions. Computers & Education 115 (2017), 126--142.

Digital Library

[13]

Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2009. The elements of statistical learning (2nd ed.). Springer Series in Statistics.

[14]

Jeffrey A. Greene, Christopher A. Oswald, and Jeffrey Pomerantz. 2015. Predictors of retention and achievement in a massive open online course. American Educational Research Journal 52, 5 (2015), 925--955.

[15]

Philip J. Guo and Katharina Reinecke. 2014. Demographic differences in how students navigate through MOOCs. In Proceedings of the first ACM conference on Learning@Scale. ACM, 21--30.

Digital Library

[16]

Sherif Halawa, Daniel Greene, and John Mitchell. 2014. Dropout prediction in MOOCs using learner activity features. Experiences and best practices in and around MOOCs 7 (2014), 3--12.

[17]

Trevor Hastie, Robert Tibshirani, and Martin Wainwright. 2015. Statistical learning with sparsity: The lasso and generalizations. CRC press.

Digital Library

[18]

Jiazhen He, James Bailey, Benjamin I. P. Rubinstein, and Rui Zhang. 2015. Identifying at-risk students in massive open online courses. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. 1749--1755.

Digital Library

[19]

Arthur E. Hoerl and Robert W. Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12, 1 (1970), 55--67.

[20]

Jason Paul Howarth, Steven D'Alessandro, Lester Johnson, and Lesley White. 2016. Learner motivation for MOOC registration and the role of MOOCs as a university 'taster'. International Journal of Lifelong Education 35, 1 (2016), 74--85.

[21]

Suhang Jiang, Adrienne Williams, Katerina Schenke, Mark Warschauer, and Diane O'dowd. 2014. Predicting MOOC performance with week 1 behavior. In Educational Data Mining 2014.

[22]

René F. Kizilcec and Sherif Halawa. 2015. Attrition and achievement gaps in online learning. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale. ACM, 57--66.

Digital Library

[23]

René F. Kizilcec, Chris Piech, and Emily Schneider. 2013. Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. In Proceedings of the third international conference on learning analytics and knowledge. ACM, 170--179.

Digital Library

[24]

Marius Kloft, Felix Stiehler, Zhilin Zheng, and Niels Pinkwart. 2014. Predicting MOOC dropout over weeks using machine learning methods. In Proceedings of the EMNLP 2014 Workshop on Analysis of Large Scale Social Interaction in MOOCs. 60--65.

[25]

Tommy Mullaney and Justin Reich. 2015. Staggered versus all-at-once content release in massive open online courses: Evaluating a natural experiment. In Proceedings of the Second (2015) ACM Conference on Learning@Scale. ACM, 185--194.

Digital Library

[26]

Jiezhong Qiu, Jie Tang, Tracy Xiao Liu, Jie Gong, Chenhui Zhang, Qian Zhang, and Yufei Xue. 2016. Modeling and predicting learning behavior in MOOCs. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining. ACM, 93--102.

Digital Library

[27]

Arti Ramesh, Dan Goldwasser, Bert Huang, Hal Daume III, and Lise Getoor. 2014. Learning latent engagement patterns of students in online courses. In Twenty-Eighth AAAI Conference on Artificial Intelligence.

Digital Library

[28]

Justin Reich. 2014. MOOC completion and retention in the context of student intent. EDUCAUSE Review Online (2014).

[29]

Carly Robinson, Michael Yeomans, Justin Reich, Chris Hulleman, and Hunter Gehlbach. 2016. Forecasting student achievement in MOOCs with natural language processing. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. ACM, 383--387.

Digital Library

[30]

Ayse Sunar, Su White, Nor Abdullah, and Hugh Davis. 2017. How learners' interactions sustain engagement: A MOOC case study. IEEE Transactions on Learning Technologies 10 (2017). Issue 4.

Digital Library

[31]

Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological) (1996), 267--288.

[32]

Miaomiao Wen, Diyi Yang, and Carolyn Penstein Rosé. 2014. Linguistic Reflections of Student Engagement in Massive Open Online Courses. In ICWSM.

[33]

Jacob Whitehill, Kiran Mohan, Daniel T. Seaton, Yigal Rosen, and Dustin Tingley. 2017. Delving Deeper into MOOC Student Dropout Prediction. CoRR abs/1702.06404 (2017). http://arxiv.org/abs/1702.06404

[34]

M. J. Wurm, P. J. Rathouz, and B. M. Hanlon. 2017. Regularized ordinal regression and the ordinalNet R package. ArXiv e-prints (June 2017).

[35]

Diyi Yang, Tanmay Sinha, David Adamson, and Carolyn Penstein Rosé. 2013. Turn on, tune in, drop out: Anticipating student dropouts in massive open online courses. In Proceedings of the 2013 NIPS Data-driven education workshop, Vol. 11. 14.

[36]

Saijing Zheng, Mary Beth Rosson, Patrick C. Shih, and John M. Carroll. 2015. Understanding student motivation, behaviors and perceptions in MOOCs. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. ACM, 1882--1895.

Digital Library

Cited By

Booth BBosch ND’Mello S(2023)Engagement Detection and Its Applications in Learning: A Tutorial and Selective ReviewProceedings of the IEEE10.1109/JPROC.2023.3309560111:10(1398-1422)Online publication date: Oct-2023
https://doi.org/10.1109/JPROC.2023.3309560
Li QSharma P(2023)The effects of learners’ background and social network position on content-related MOOC interactionEducational technology research and development10.1007/s11423-023-10221-471:3(973-990)Online publication date: 25-Apr-2023
https://doi.org/10.1007/s11423-023-10221-4
UÇAR ASARITEPECİ M(2022)Kitlesel Açık Çevrimiçi Ders Ortamlarında Öğrenci KatılımıÖğretim Teknolojisi ve Hayat Boyu Öğrenme Dergisi - Instructional Technology and Lifelong Learning10.52911/itall.1194260Online publication date: 11-Dec-2022
https://doi.org/10.52911/itall.1194260
Show More Cited By

Recommendations

Promoting Student Engagement in MOOCs
L@S '16: Proceedings of the Third (2016) ACM Conference on Learning @ Scale

MOOCs offer valuable learning experiences to students from all around the world. In addition to providing filmed lectures, readings, and problem sets, many MOOCs allow students to ask and answer questions about course materials with each other through ...
Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement
Design for Teaching and Learning in a Networked World
Abstract
Massive open online courses (MOOCs) are part of the lifelong learning experience of people worldwide. Many of these learners participate fully. However, the high levels of dropout on most of these courses are a cause for concern. Previous studies ...
Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs)
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge

Massive open online courses (MOOCs) are now being used across the world to provide millions of learners with access to education. Many learners complete these courses successfully, or to their own satisfaction, but the high numbers who do not finish ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

L@S '18: Proceedings of the Fifth Annual ACM Conference on Learning at Scale

June 2018

391 pages

ISBN:9781450358866

DOI:10.1145/3231644

Conference Chair:
Rose Luckin
UCL Institute of Education, UK
,
Program Chairs:
Scott Klemmer
University of California at San Diego
,
Kenneth Koedinger
Carnegie Mellon University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

L@S '18

L@S '18: Fifth (2018) ACM Conference on Learning @ Scale

June 26 - 28, 2018

London, United Kingdom

Acceptance Rates

L@S '18 Paper Acceptance Rate 24 of 58 submissions, 41%;

Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
398
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Booth BBosch ND’Mello S(2023)Engagement Detection and Its Applications in Learning: A Tutorial and Selective ReviewProceedings of the IEEE10.1109/JPROC.2023.3309560111:10(1398-1422)Online publication date: Oct-2023
https://doi.org/10.1109/JPROC.2023.3309560
Li QSharma P(2023)The effects of learners’ background and social network position on content-related MOOC interactionEducational technology research and development10.1007/s11423-023-10221-471:3(973-990)Online publication date: 25-Apr-2023
https://doi.org/10.1007/s11423-023-10221-4
UÇAR ASARITEPECİ M(2022)Kitlesel Açık Çevrimiçi Ders Ortamlarında Öğrenci KatılımıÖğretim Teknolojisi ve Hayat Boyu Öğrenme Dergisi - Instructional Technology and Lifelong Learning10.52911/itall.1194260Online publication date: 11-Dec-2022
https://doi.org/10.52911/itall.1194260
Akhuseyinoglu KBrusilovsky P(2022)Exploring Behavioral Patterns for Data-Driven Modeling of Learners' Individual DifferencesFrontiers in Artificial Intelligence10.3389/frai.2022.8073205Online publication date: 15-Feb-2022
https://doi.org/10.3389/frai.2022.807320
Yang BHe HKizilcec RDavis KOchoa X(2022)Learning Alone Yet Together: Enhancing Between-Learner Social Connectivity at ScaleProceedings of the Ninth ACM Conference on Learning @ Scale10.1145/3491140.3528294(374-378)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1145/3491140.3528294
Liu SLiu SLiu ZPeng XYang Z(2022)Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievementComputers & Education10.1016/j.compedu.2022.104461181:COnline publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.compedu.2022.104461
Akhuseyinoglu KBrusilovsky P(2021)Data-Driven Modeling of Learners’ Individual Differences for Predicting Engagement and Success in Online LearningProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450613.3456834(201-212)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450613.3456834
Gledson AApaolaza ABarthold SGünther FYu HVigo M(2021)Characterising Student Engagement Modes through Low-Level Activity PatternsProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450613.3456818(88-97)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450613.3456818
Luo VKlinkert LFoster PTseng CAdams EKetterlin-Geller LLarson EClark C(2021)A Multidisciplinary Approach To Designing Immersive Gameplay Elements for Learning Standard-Based Educational ContentExtended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play10.1145/3450337.3483467(67-73)Online publication date: 15-Oct-2021
https://dl.acm.org/doi/10.1145/3450337.3483467
Zhou JBhat SScheffel MDowell NJoksimovic SSiemens G(2021)Modeling Consistency Using Engagement Patterns in Online CoursesLAK21: 11th International Learning Analytics and Knowledge Conference10.1145/3448139.3448161(226-236)Online publication date: 12-Apr-2021
https://dl.acm.org/doi/10.1145/3448139.3448161
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten