Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Characterizing Student Engagement Moods for Dropout Prediction in Question Pool Websites

Published: 22 April 2021 Publication History

Abstract

Problem-Based Learning (PBL) is a popular approach to instruction that supports students to get hands-on training by solving problems. Question Pool websites (QPs) such as LeetCode, Code Chef, and Math Playground help PBL by supplying authentic, diverse, and contextualized questions to students. Nonetheless, empirical findings suggest that 40% to 80% of students registered in QPs drop out in less than two months. This research is the first attempt to understand and predict student dropouts from QPs via exploiting students' engagement moods. Adopting a data-driven approach, we identify five different engagement moods for QP students, which are namely challenge-seeker, subject-seeker, interest-seeker, joy-seeker, and non-seeker. We find that students have collective preferences for answering questions in each engagement mood, and deviation from those preferences increases their probability of dropping out significantly. Last but not least, this paper contributes by introducing a new hybrid machine learning model (we call Dropout-Plus) for predicting student dropouts in QPs. The test results on a popular QP in China, with nearly 10K students, show that Dropout-Plus can exceed the rival algorithms' dropout prediction performance in terms of accuracy, F1-measure, and AUC. We wrap up our work by giving some design suggestions to QP managers and online learning professionals to reduce their student dropouts.

References

[1]
Najwa Alghamdi, Nora Alrajebah, and Shiroq Al-Megren. 2019. Crowd Behavior Analysis Using Snap Map: A Preliminary Study on the Grand Holy Mosque in Mecca. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing (Austin, TX, USA) (CSCW '19). Association for Computing Machinery, New York, NY, USA, 137--141. https://doi.org/10.1145/3311957.3359473
[2]
Nese Alyuz, Eda Okur, Utku Genc, Sinem Aslan, Cagri Tanriover, and Asli Arslan Esme. 2017. An Unobtrusive and Multimodal Approach for Behavioral Engagement Detection of Students. In Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education (Glasgow, UK) (MIE 2017). Association for Computing Machinery, New York, NY, USA, 26--32. https://doi.org/10.1145/3139513.3139521
[3]
Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2013. Steering User Behavior with Badges. In Proceedings of the 22nd International Conference on World Wide Web (Rio de Janeiro, Brazil) (WWW '13). Association for Computing Machinery, New York, NY, USA, 95--106. https://doi.org/10.1145/2488388.2488398
[4]
Sinem Aslan, Nese Alyuz, Cagri Tanriover, Sinem E. Mete, Eda Okur, Sidney K. D'Mello, and Asli Arslan Esme. 2019. Investigating the Impact of a Real-Time, Multimodal Student Engagement Analytics Technology in Authentic Classrooms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300534
[5]
R. Babbie. 2016. The Basics of Social Research. Cengage Learning. 2015936245 https://books.google.com/books?id=0CJTCwAAQBAJ
[6]
Ryan Baker, Jason Walonoski, Neil Heffernan, Ido Roll, Albert Corbett, and Kenneth Koedinger. 2008. Why students engage in "gaming the system" behavior in interactive learning environments. Journal of Interactive Learning Research, Vol. 19, 2 (2008), 185--224.
[7]
Ryan Shaun Baker. 2005. Designing intelligent tutors that adapt to when students game the system. Ph.D. Dissertation. Carnegie Mellon University Pittsburgh.
[8]
Ryan S.J.d. Baker. 2007. Modeling and Understanding Students' off-Task Behavior in Intelligent Tutoring Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '07). Association for Computing Machinery, New York, NY, USA, 1059--1068. https://doi.org/10.1145/1240624.1240785
[9]
Ryan Shaun Baker, Albert T. Corbett, Kenneth R. Koedinger, and Angela Z. Wagner. 2004. Off-Task Behavior in the Cognitive Tutor Classroom: When Students "Game the System". In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI '04). Association for Computing Machinery, New York, NY, USA, 383--390. https://doi.org/10.1145/985692.985741
[10]
E.F. Barkley. 2009. Student Engagement Techniques: A Handbook for College Faculty. Wiley. 2009029117 https://books.google.com/books?id=muAStyrwyZgC
[11]
Alexander Bartel, Paula Figas, and Georg Hagel. 2015. Towards a Competency-Based Education with Gamification Design Elements. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (London, United Kingdom) (CHI PLAY '15). Association for Computing Machinery, New York, NY, USA, 457--462. https://doi.org/10.1145/2793107.2810325
[12]
Jonathan Bassen, Bharathan Balaji, Michael Schaarschmidt, Candace Thille, Jay Painter, Dawn Zimmaro, Alex Games, Ethan Fast, and John C. Mitchell. 2020. Reinforcement Learning for the Adaptive Scheduling of Educational Activities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376518
[13]
Leonard E. Baum, Ted Petrie, George Soules, and Norman Weiss. 1970. A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains. The Annals of Mathematical Statistics, Vol. 41, 1 (1970), 164--171. http://www.jstor.org/stable/2239727
[14]
Johann Benerradi, Horia A. Maior, Adrian Marinescu, Jeremie Clos, and Max L. Wilson. 2019. Exploring Machine Learning Approaches for Classifying Mental Workload Using FNIRS Data from HCI Tasks. In Proceedings of the Halfway to the Future Symposium 2019 (Nottingham, United Kingdom) (HTTF 2019). Association for Computing Machinery, New York, NY, USA, Article 8, bibinfonumpages11 pages. https://doi.org/10.1145/3363384.3363392
[15]
Muhammad Ilyas Bhutto. 2011. Effects of Social Reinforcers on Students'Learning Outcomes at Secondary School Level. International Journal of Academic Research in Business and Social Sciences, Vol. 1, 2 (2011), 71.
[16]
Christopher M Bishop. 2006. Machine learning and pattern recognition. Information science and statistics. Springer, Heidelberg (2006).
[17]
Nigel Bosch. 2016. Detecting Student Engagement: Human Versus Machine. In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (Halifax, Nova Scotia, Canada) (UMAP '16). Association for Computing Machinery, New York, NY, USA, 317--320. https://doi.org/10.1145/2930238.2930371
[18]
Leo Breiman. 2001. Random forests. Machine Learning, Vol. 45, 1 (2001), 5--32. https://doi.org/10.1023/a:1010933404324
[19]
Elise Cappella, Ha Yeon Kim, Jennifer W. Neal, and Daisy R. Jackson. 2013. Classroom Peer Relationships and Behavioral Engagement in Elementary School: The Role of Social Network Equity. American Journal of Community Psychology, Vol. 52, 3--4 (Oct. 2013), 367--379. https://doi.org/10.1007/s10464-013-9603-5
[20]
Jonathan Carlton, Andy Brown, Caroline Jay, and John Keane. 2019. Inferring User Engagement from Interaction Data. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3290607.3313009
[21]
Xunru Che, Danaë Metaxa-Kakavouli, and Jeffrey T. Hancock. 2018. Fake News in the News: An Analysis of Partisan Coverage of the Fake News Phenomenon. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (Jersey City, NJ, USA) (CSCW '18). Association for Computing Machinery, New York, NY, USA, 289--292. https://doi.org/10.1145/3272973.3274079
[22]
Code Chef. 2019. Code Chef. https://www.codechef.com/ Retrieved January 24, 2019 from
[23]
Scott Chen, Ponani Gopalakrishnan, et almbox. 1998. Speaker, environment and channel change detection and clustering via the bayesian information criterion. In Proc. DARPA broadcast news transcription and understanding workshop, Vol. 8. Virginia, USA, 127--132.
[24]
Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 785--794. https://doi.org/10.1145/2939672.2939785
[25]
Yuanzhe Chen, Qing Chen, Mingqian Zhao, Sebastien Boyer, Kalyan Veeramachaneni, and Huamin Qu. 2016. DropoutSeer: Visualizing learning patterns in Massive Open Online Courses for dropout reasoning and prediction. In 2016 IEEE Conference on Visual Analytics Science and Technology (VAST). IEEE, 111--120.
[26]
Yinghan Chen, Steven Andrew Culpepper, Shiyu Wang, and Jeffrey Douglas. 2017. A Hidden Markov Model for Learning Trajectories in Cognitive Diagnosis With Application to Spatial Rotation Skills. Applied Psychological Measurement, Vol. 42, 1 (Sept. 2017), 5--23. https://doi.org/10.1177/0146621617721250
[27]
Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1217--1230. https://doi.org/10.1145/2998181.2998213
[28]
Chia-Fang Chung, Nanna Gorm, Irina A. Shklovski, and Sean Munson. 2017. Finding the Right Fit: Understanding Health Tracking in Workplace Wellness Programs. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 4875--4886. https://doi.org/10.1145/3025453.3025510
[29]
Evandro B. Costa, Baldoino Fonseca, Marcelo Almeida Santana, Fabrísia Ferreira de Araújo, and Joilson Rego. 2017. Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses. Computers in Human Behavior, Vol. 73 (2017), 247--256. https://doi.org/10.1016/j.chb.2017.01.047
[30]
M Csikszentmihalyi. 1975. Beyond boredom and anxiety. San Francisco: JosseyBass. Well-being: Thefoundations of hedonic psychology (1975), 134--154.
[31]
Ryan S. J. d. Baker, Albert T. Corbett, Ido Roll, and Kenneth R. Koedinger. 2008. Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, Vol. 18, 3 (Jan. 2008), 287--314. https://doi.org/10.1007/s11257-007-9045-6
[32]
Matt Dixon, Nalin Asanka Gamagedara Arachchilage, and James Nicholson. 2019. Engaging Users with Educational Games: The Case of Phishing. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3290607.3313026
[33]
Edson B. dos Santos Junior, Carlos Sim oes, Ana Cristina Bicharra Garcia, and Adriana S. Vivacqua. 2018. What Does a Crowd Routing Behavior Change Reveal?. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (Jersey City, NJ, USA) (CSCW '18). Association for Computing Machinery, New York, NY, USA, 297--300. https://doi.org/10.1145/3272973.3274081
[34]
Gideon Dror, Dan Pelleg, Oleg Rokhlenko, and Idan Szpektor. 2012. Churn Prediction in New Users of Yahoo! Answers. In Proceedings of the 21st International Conference on World Wide Web (Lyon, France) (WWW '12 Companion). ACM, New York, NY, USA, 829--834. https://doi.org/10.1145/2187980.2188207
[35]
Louis Faucon, Lukasz Kidzinski, and Pierre Dillenbourg. 2016. Semi-Markov Model for Simulating MOOC Students. International Educational Data Mining Society (2016).
[36]
Sarah Foley, Nadia Pantidi, and John McCarthy. 2020. Student Engagement in Sensitive Design Contexts: A Case Study in Dementia Care. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376161
[37]
G David Forney Jr. 2005. The viterbi algorithm: A personal history. arXiv preprint cs/0504020 (2005). http://arxiv.org/abs/cs/0504020
[38]
Pascal E. Fortin, Elisabeth Sulmont, and Jeremy Cooperstock. 2019. Detecting Perception of Smartphone Notifications Using Skin Conductance Responses. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3290605.3300420
[39]
Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research, Vol. 74, 1 (March 2004), 59--109. https://doi.org/10.3102/00346543074001059
[40]
Adabriand Furtado, Nazareno Andrade, Nigini Oliveira, and Francisco Brasileiro. 2013. Contributor Profiles, Their Dynamics, and Their Importance in Five Q&a Sites. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (San Antonio, Texas, USA) (CSCW '13). Association for Computing Machinery, New York, NY, USA, 1237--1252. https://doi.org/10.1145/2441776.2441916
[41]
W.L. Gardiner. 1974. Psychology: a story of a search .Brooks/Cole Pub. Co. 73089595 https://books.google.fr/books?id=q50bAQAAMAAJ
[42]
Nate Gruver, Ali Malik, Brahm Capoor, Chris Piech, Mitchell L Stevens, and Andreas Paepcke. 2019. Using Latent Variable Models to Observe Academic Pathways. In Proceedings of The 12th International Conference on Educational Data Mining. EDM, 294--299. http://educationaldatamining.org/edm2019/proceedings/
[43]
Juho Hamari, David J. Shernoff, Elizabeth Rowe, Brianno Coller, Jodi Asbell-Clarke, and Teon Edwards. 2016. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, Vol. 54 (2016), 170--179. https://doi.org/10.1016/j.chb.2015.07.045
[44]
Siti Suhaila Abdul Hamid, Novia Admodisastro, Noridayu Manshor, Azrina Kamaruddin, and Abdul Azim Abd Ghani. 2018. Dyslexia adaptive learning model: student engagement prediction using machine learning approach. In International Conference on Soft Computing and Data Mining. Springer, 372--384.
[45]
Mitchell M Handelsman, William L Briggs, Nora Sullivan, and Annette Towler. 2005. A measure of college student course engagement. The Journal of Educational Research, Vol. 98, 3 (2005), 184--192.
[46]
Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P. Bigham. 2018. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, Article 449, bibinfonumpages14 pages. https://doi.org/10.1145/3173574.3174023
[47]
Xiaofei He, Xinbo Gao, Yanning Zhang, Zhi-Hua Zhou, Zhi-Yong Liu, Baochuan Fu, Fuyuan Hu, and Zhancheng Zhang. 2015. Intelligence Science and Big Data Engineering. Big Data and Machine Learning Techniques: 5th International Conference, IScIDE 2015, Suzhou, China, June 14--16, 2015, Revised Selected Papers. Vol. 9243. Springer.
[48]
Matthew Hertz. 2010. What Do "CS1" and "CS2" Mean? Investigating Differences in the Early Courses. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education (Milwaukee, Wisconsin, USA) (SIGCSE '10). Association for Computing Machinery, New York, NY, USA, 199--203. https://doi.org/10.1145/1734263.1734335
[49]
Hayati Hind, Mohammed Khalidi Idrissi, and Samir Bennani. 2017. Applying Text Mining to Predict Learners' Cognitive Engagement. In Proceedings of the Mediterranean Symposium on Smart City Application (Tangier, Morocco) (SCAMS '17). Association for Computing Machinery, New York, NY, USA, Article 2, bibinfonumpages6 pages. https://doi.org/10.1145/3175628.3175655
[50]
Beryl Hoffman, Ralph Morelli, and Jennifer Rosato. 2019. Student Engagement is Key to Broadening Participation in CS. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE '19). ACM, New York, NY, USA, 1123--1129. https://doi.org/10.1145/3287324.3287438
[51]
Jeroen Janssen and Paul A Kirschner. 2020. Applying collaborative cognitive load theory to computer-supported collaborative learning: towards a research agenda. Educational Technology Research and Development (2020), 1--23.
[52]
Kenneth Joseph, Wei Wei, and Kathleen M. Carley. 2017. Girls Rule, Boys Drool: Extracting Semantic and Affective Stereotypes from Twitter. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1362--1374. https://doi.org/10.1145/2998181.2998187
[53]
Libor Juha?ák, Ji?í Zounek, and Lucie Rohlíková. 2019. Using process mining to analyze students' quiz-taking behavior patterns in a learning management system. Computers in Human Behavior, Vol. 92 (2019), 496--506. https://doi.org/10.1016/j.chb.2017.12.015
[54]
Jutge. 2019. jutge. https://jutge.org/ Retrieved January 24, 2019 from
[55]
Saskia M. Kelders and Hanneke Kip. 2019. Development and Initial Validation of a Scale to Measure Engagement with EHealth Technologies. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3290607.3312917
[56]
Jina Kim, Kunwoo Bae, Eunil Park, and Angel P. del Pobil. 2019. Who Will Subscribe to My Streaming Channel? The Case of Twitch. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing (Austin, TX, USA) (CSCW '19). Association for Computing Machinery, New York, NY, USA, 247--251. https://doi.org/10.1145/3311957.3359470
[57]
Juho Kim, Philip J. Guo, Daniel T. Seaton, Piotr Mitros, Krzysztof Z. Gajos, and Robert C. Miller. 2014. Understanding In-Video Dropouts and Interaction Peaks Inonline Lecture Videos. In Proceedings of the First ACM Conference on Learning @ Scale Conference (Atlanta, Georgia, USA) (L@S '14). Association for Computing Machinery, New York, NY, USA, 31--40. https://doi.org/10.1145/2556325.2566237
[58]
Marios Kokkodis. 2019. Reputation Deflation Through Dynamic Expertise Assessment in Online Labor Markets. In The World Wide Web Conference (San Francisco, CA, USA) (WWW '19). ACM, New York, NY, USA, 896--905. https://doi.org/10.1145/3308558.3313479
[59]
George D Kuh. 2001. Assessing what really matters to student learning inside the national survey of student engagement. Change: The Magazine of Higher Learning, Vol. 33, 3 (2001), 10--17.
[60]
Il-Youp Kwak, Jun Ho Huh, Seung Taek Han, Iljoo Kim, and Jiwon Yoon. 2019. Voice Presentation Attack Detection through Text-Converted Voice Command Analysis. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300828
[61]
Young D. Kwon, Dimitris Chatzopoulos, Ehsan ul Haq, Raymond Chi-Wing Wong, and Pan Hui. 2019. GeoLifecycle: User Engagement of Geographical Exploration and Churn Prediction in LBSNs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 3, 3, Article 92 (Sept. 2019), bibinfonumpages29 pages. https://doi.org/10.1145/3351250
[62]
D Langley. 2006. The student engagement index: A proposed student rating system based on the national benchmarks of effective educational practice. University of Minnesota: Center for Teaching and Learning Services (2006).
[63]
Hyunsoo Lee, Uichin Lee, and Hwajung Hong. 2019. Commitment Devices in Online Behavior Change Support Systems. In Proceedings of Asian CHI Symposium 2019: Emerging HCI Research Collection (Glasgow, Scotland, United Kingdom) (AsianHCI '19). Association for Computing Machinery, New York, NY, USA, 105--113. https://doi.org/10.1145/3309700.3338446
[64]
LeetCode. 2019. LeetCode. https://leetcode.com/ Retrieved January 24, 2019 from
[65]
Pascal Lessel, Maximilian Altmeyer, Lea Verena Schmeer, and Antonio Krüger. 2019. "Enable or Disable Gamification?": Analyzing the Impact of Choice in a Gamified Image Tagging Task. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300380
[66]
Zipeng Liu, Zhicheng Liu, and Tamara Munzner. 2020. Data-Driven Multi-Level Segmentation of Image Editing Logs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376152
[67]
Suman Kalyan Maity, Aishik Chakraborty, Pawan Goyal, and Animesh Mukherjee. 2017. Detection of Sockpuppets in Social Media. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 243--246. https://doi.org/10.1145/3022198.3026360
[68]
Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, René F. Kizilcec, Nicolás Morales, and Jorge Munoz-Gama. 2018. Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in Massive Open Online Courses. Computers in Human Behavior, Vol. 80 (2018), 179--196. https://doi.org/10.1016/j.chb.2017.11.011
[69]
Hiroaki Masaki, Kengo Shibata, Shui Hoshino, Takahiro Ishihama, Nagayuki Saito, and Koji Yatani. 2020. Exploring Nudge Designs to Help Adolescent SNS Users Avoid Privacy and Safety Threats. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--11. https://doi.org/10.1145/3313831.3376666
[70]
Miller McPherson, Lynn Smith-Lovin, and James M Cook. 2001. Birds of a feather: Homophily in social networks. Annual review of sociology, Vol. 27, 1 (2001), 415--444.
[71]
Pardis Miri, Emily Jusuf, Andero Uusberg, Horia Margarit, Robert Flory, Katherine Isbister, Keith Marzullo, and James J. Gross. 2020. Evaluating a Personalizable, Inconspicuous Vibrotactile(PIV) Breathing Pacer for In-the-Moment Affect Regulation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376757
[72]
Reza Hadi Mogavi, Sujit Gujar, Xiaojuan Ma, and Pan Hui. 2019. HRCR: Hidden Markov-Based Reinforcement to Reduce Churn in Question Answering Forums. In PRICAI 2019: Trends in Artificial Intelligence. Springer International Publishing, 364--376. https://doi.org/10.1007/978-3-030-29908-8_29
[73]
Michael Morgan, Matthew Butler, Neena Thota, and Jane Sinclair. 2018. How CS Academics View Student Engagement. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (Larnaca, Cyprus) (ITiCSE 2018). ACM, New York, NY, USA, 284--289. https://doi.org/10.1145/3197091.3197092
[74]
Abdallah Moubayed, Mohammadnoor Injadat, Abdallah Shami, and Hanan Lutfiyya. 2020. Student Engagement Level in an e-Learning Environment: Clustering Using K-means. American Journal of Distance Education, Vol. 34, 2 (March 2020), 137--156. https://doi.org/10.1080/08923647.2020.1696140
[75]
Benjamin Murauer and Günther Specht. 2018. Detecting Music Genre Using Extreme Gradient Boosting. In Companion Proceedings of the The Web Conference 2018 (Lyon, France) (WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1923--1927. https://doi.org/10.1145/3184558.3191822
[76]
Saurabh Nagrecha, John Z. Dillon, and Nitesh V. Chawla. 2017. MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable. In Proceedings of the 26th International Conference on World Wide Web Companion (Perth, Australia) (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 351--359. https://doi.org/10.1145/3041021.3054162
[77]
Junpei Naito, Yukino Baba, Hisashi Kashima, Takenori Takaki, and Takuya Funo. 2018. Predictive modeling of learning continuation in preschool education using temporal patterns of development tests. In Thirty-Second AAAI Conference on Artificial Intelligence.
[78]
Tuan Dinh Nguyen, Marisa Cannata, and Jason Miller. 2016. Understanding student behavioral engagement: Importance of student interaction with peers and teachers. The Journal of Educational Research, Vol. 111, 2 (Oct. 2016), 163--174. https://doi.org/10.1080/00220671.2016.1220359
[79]
Kenji Okamoto. 2017. Analyzing Single Molecule FRET Trajectories Using HMM. In Hidden Markov Models. Springer New York, 103--113. https://doi.org/10.1007/978-1-4939-6753-7_7
[80]
Judith A Ouimet and Robert A Smallwood. 2005. Assessment Measures: CLASSE--The Class-Level Survey of Student Engagement. Assessment Update, Vol. 17, 6 (2005), 13--15.
[81]
Steven Pace. 2004. A grounded theory of the flow experiences of Web users. International Journal of Human-Computer Studies, Vol. 60, 3 (2004), 327--363. https://doi.org/10.1016/j.ijhcs.2003.08.005
[82]
Zilong Pan, Chenglu Li, and Min Liu. 2020. Learning Analytics Dashboard for Problem-Based Learning. In Proceedings of the Seventh ACM Conference on Learning @ Scale (Virtual Event, USA) (L@S '20). Association for Computing Machinery, New York, NY, USA, 393--396. https://doi.org/10.1145/3386527.3406751
[83]
Sira Park, Susan D. Holloway, Amanda Arendtsz, Janine Bempechat, and Jin Li. 2011. What Makes Students Engaged in Learning? A Time-Use Study of Within- and Between-Individual Predictors of Emotional Engagement in Low-Performing High Schools. Journal of Youth and Adolescence, Vol. 41, 3 (Dec. 2011), 390--401. https://doi.org/10.1007/s10964-011-9738-3
[84]
Sophie Parsons, Peter M. Atkinson, Elena Simperl, and Mark Weal. 2015. Thematically Analysing Social Network Content During Disasters Through the Lens of the Disaster Management Lifecycle. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15 Companion). ACM, New York, NY, USA, 1221--1226. https://doi.org/10.1145/2740908.2741721
[85]
Filipe D. Pereira, Elaine Oliveira, Alexandra Cristea, David Fernandes, Luciano Silva, Gene Aguiar, Ahmed Alamri, and Mohammad Alshehri. 2019. Early Dropout Prediction for Programming Courses Supported by Online Judges. In Lecture Notes in Computer Science. Springer International Publishing, 67--72. https://doi.org/10.1007/978-3-030-23207-8_13
[86]
Jordi Petit, Omer Giménez, and Salvador Roura. 2012. Jutge.Org: An Educational Programming Judge. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (Raleigh, North Carolina, USA) (SIGCSE '12). ACM, New York, NY, USA, 445--450. https://doi.org/10.1145/2157136.2157267
[87]
Math Playground. 2019. Math Playground. https://www.mathplayground.com/ Retrieved January 24, 2019 from
[88]
Jagat Sastry Pudipeddi, Leman Akoglu, and Hanghang Tong. 2014. User Churn in Focused Question Answering Sites: Characterizations and Prediction. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW '14 Companion). ACM, New York, NY, USA, 469--474. https://doi.org/10.1145/2567948.2576965
[89]
Jiezhong Qiu, Jie Tang, Tracy Xiao Liu, Jie Gong, Chenhui Zhang, Qian Zhang, and Yufei Xue. 2016. Modeling and Predicting Learning Behavior in MOOCs. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (San Francisco, California, USA) (WSDM '16). Association for Computing Machinery, New York, NY, USA, 93--102. https://doi.org/10.1145/2835776.2835842
[90]
Rezvaneh Rezapour and Jana Diesner. 2017. Classification and Detection of Micro-Level Impact of Issue-Focused Documentary Films Based on Reviews. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1419--1431. https://doi.org/10.1145/2998181.2998201
[91]
Juan D Rodriguez, Aritz Perez, and Jose A Lozano. 2009. Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE transactions on pattern analysis and machine intelligence, Vol. 32, 3 (2009), 569--575.
[92]
Cristó bal Romero and Sebastiá n Ventura. 2016. Educational data science in massive open online courses. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 7, 1 (Sept. 2016), e1187. https://doi.org/10.1002/widm.1187
[93]
Matthew Rowe. 2013. Mining user lifecycles from online community platforms and their application to churn prediction. In 2013 IEEE 13th International Conference on Data Mining. IEEE, 637--646.
[94]
Victor B. Saenz, Deryl Hatch, Beth E. Bukoski, Suyun Kim, Kye hyoung Lee, and Patrick Valdez. 2011. Community College Student Engagement Patterns. Community College Review, Vol. 39, 3 (July 2011), 235--267. https://doi.org/10.1177/0091552111416643
[95]
Koya Sato, Mizuki Oka, and Kazuhiko Kato. 2019. Early Churn User Classification in Social Networking Service Using Attention-Based Long Short-Term Memory. In Lecture Notes in Computer Science. Springer International Publishing, 45--56. https://doi.org/10.1007/978-3-030-26142-9_5
[96]
Junfeng Shang and Joseph E. Cavanaugh. 2008. Bootstrap variants of the Akaike information criterion for mixed model selection. Computational Statistics & Data Analysis, Vol. 52, 4 (2008), 2004--2021. https://doi.org/10.1016/j.csda.2007.06.019
[97]
David J Shernoff and Mihaly Csikszentmihalyi. 2009. Cultivating engaged learners and optimal learning environments. Handbook of positive psychology in schools (2009), 131--145.
[98]
Hongzhi Shi, Chao Zhang, Quanming Yao, Yong Li, Funing Sun, and Depeng Jin. 2019. State-Sharing Sparse Hidden Markov Models for Personalized Sequences. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Anchorage, AK, USA) (KDD '19). Association for Computing Machinery, New York, NY, USA, 1549--1559. https://doi.org/10.1145/3292500.3330828
[99]
Hyunjin Shin, Bugeun Kim, and Gahgene Gweon. 2020. Guessing or Solving? Exploring the Use of Motion Features from Educational Game Logs. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8. https://doi.org/10.1145/3334480.3383005
[100]
Manya Sleeper, Alessandro Acquisti, Lorrie Faith Cranor, Patrick Gage Kelley, Sean A. Munson, and Norman Sadeh. 2015. I Would Like To..., I Shouldn't..., I Wish I...: Exploring Behavior-Change Goals for Social Networking Sites. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW '15). Association for Computing Machinery, New York, NY, USA, 1058--1069. https://doi.org/10.1145/2675133.2675193
[101]
A. Spanos. 2019. Probability Theory and Statistical Inference: Empirical Modeling with Observational Data. Cambridge University Press. https://books.google.com/books?id=nm_IDwAAQBAJ
[102]
Ivan Srba and Maria Bielikova. 2016. A Comprehensive Survey and Classification of Approaches for Community Question Answering. ACM Trans. Web, Vol. 10, 3, Article 18 (Aug. 2016), 63 pages. https://doi.org/10.1145/2934687
[103]
Cor JM Suhre, Ellen PWA Jansen, and Egbert G Harskamp. 2007. Impact of degree program satisfaction on the persistence of college students. Higher Education, Vol. 54, 2 (2007), 207--226.
[104]
Stephanie D. Teasley. 2017. Student Facing Dashboards: One Size Fits All? Technology, Knowledge and Learning, Vol. 22, 3 (01 Oct 2017), 377--384. https://doi.org/10.1007/s10758-017-9314-3
[105]
Jacob Thebault-Spieker, Anbang Xu, Jilin Chen, Jalal Mahmud, and Jeffrey Nichols. 2016. Exploring Engagement in a 'Social Crowd' on Twitter. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 417--420. https://doi.org/10.1145/2818052.2869112
[106]
Timus. 2019. Timus Online Judge. https://acm.timus.ru/ Retrieved January 24, 2019 from
[107]
Gustavo F. Tondello, Dennis L. Kappen, Marim Ganaba, and Lennart E. Nacke. 2019. Gameful Design Heuristics: A Gamification Inspection Tool. In Human-Computer Interaction. Perspectives on Design. Springer International Publishing, 224--244. https://doi.org/10.1007/978-3-030-22646-6_16
[108]
Gustavo F. Tondello, Rina R. Wehbe, Lisa Diamond, Marc Busch, Andrzej Marczewski, and Lennart E. Nacke. 2016. The Gamification User Types Hexad Scale. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (Austin, Texas, USA) (CHI PLAY '16). Association for Computing Machinery, New York, NY, USA, 229--243. https://doi.org/10.1145/2967934.2968082
[109]
Richard Tucker. 2016. Collaboration and Student Engagement in Design Education .IGI Global.
[110]
T. Vafeiadis, K.I. Diamantaras, G. Sarigiannidis, and K.Ch. Chatzisavvas. 2015. A comparison of machine learning techniques for customer churn prediction. Simulation Modelling Practice and Theory, Vol. 55 (2015), 1--9. https://doi.org/10.1016/j.simpat.2015.03.003
[111]
Wil Van Der Aalst. 2011. Process mining: discovery, conformance and enhancement of business processes. Vol. 2. Springer.
[112]
Devin Waldrop, Amy L. Reschly, Kathleen Fraysier, and James J. Appleton. 2018. Measuring the Engagement of College Students: Administration Format, Structure, and Validity of the Student Engagement InstrumenttextendashCollege. Measurement and Evaluation in Counseling and Developmen, Vol. 52, 2 (Nov. 2018), 90--107. https://doi.org/10.1080/07481756.2018.1497429
[113]
Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019 b. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--15. https://doi.org/10.1145/3290605.3300831
[114]
Qianwen Wang, Yao Ming, Zhihua Jin, Qiaomu Shen, Dongyu Liu, Micah J. Smith, Kalyan Veeramachaneni, and Huamin Qu. 2019 a. ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300911
[115]
Wei Wang, Han Yu, and Chunyan Miao. 2017. Deep Model for Dropout Prediction in MOOCs. In Proceedings of the 2nd International Conference on Crowd Science and Engineering (Beijing, China) (ICCSE'17). Association for Computing Machinery, New York, NY, USA, 26--32. https://doi.org/10.1145/3126973.3126990
[116]
Vicky Ward, Allan House, and Susan Hamer. 2009. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. Journal of health services research & policy, Vol. 14, 3 (2009), 156--164.
[117]
Szymon Wasik, Maciej Antczak, Jan Badura, Artur Laskowski, and Tomasz Sternal. 2016. Optil.Io: Cloud Based Platform For Solving Optimization Problems Using Crowdsourcing Approach. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 433--436. https://doi.org/10.1145/2818052.2869098
[118]
Szymon Wasik, Maciej Antczak, Jan Badura, Artur Laskowski, and Tomasz Sternal. 2018. A Survey on Online Judge Systems and Their Applications. ACM Comput. Surv., Vol. 51, 1, Article 3 (Jan. 2018), bibinfonumpages34 pages. https://doi.org/10.1145/3143560
[119]
Jacob Whitehill, Kiran Mohan, Daniel Seaton, Yigal Rosen, and Dustin Tingley. 2017. MOOC Dropout Prediction: How to Measure Accuracy?. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale (Cambridge, Massachusetts, USA) (L@S '17). Association for Computing Machinery, New York, NY, USA, 161--164. https://doi.org/10.1145/3051457.3053974
[120]
Meng Xia, Mingfei Sun, Huan Wei, Qing Chen, Yong Wang, Lei Shi, Huamin Qu, and Xiaojuan Ma. 2019. PeerLens: Peer-inspired Interactive Learning Path Planning in Online Question Pool. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). ACM, New York, NY, USA, Article 634, bibinfonumpages12 pages. https://doi.org/10.1145/3290605.3300864
[121]
Wanli Xing and Dongping Du. 2018. Dropout Prediction in MOOCs: Using Deep Learning for Personalized Intervention. Journal of Educational Computing Research, Vol. 57, 3 (March 2018), 547--570. https://doi.org/10.1177/0735633118757015
[122]
Yuan Xuan, Yang Chen, Huiying Li, Pan Hui, and Lei Shi. 2016. LBSNShield: Malicious Account Detection in Location-Based Social Networks. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 437--440. https://doi.org/10.1145/2818052.2869094
[123]
Carl Yang, Xiaolin Shi, Luo Jie, and Jiawei Han. 2018b. I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application. In Proceedings of the 24th ACM SIGKDD (London, United Kingdom) (KDD '18). ACM, New York, NY, USA, 914--922. https://doi.org/10.1145/3219819.3219821
[124]
Yang Yang, Zongtao Liu, Chenhao Tan, Fei Wu, Yueting Zhuang, and Yafeng Li. 2018a. To Stay or to Leave: Churn Prediction for Urban Migrants in the Initial Period. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 967--976. https://doi.org/10.1145/3178876.3186144
[125]
Zamzami Zainuddin, Samuel Kai Wah Chu, Muhammad Shujahat, and Corinne Jacqueline Perera. 2020. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review (2020), 100326.
[126]
Jiawei Zhang, Xiangnan Kong, and S Yu Philip. 2016. Social badge system analysis. In 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 453--460.
[127]
Xi Zhang, Shan Jiang, and Yihang Cheng. 2017. Inferring the Student Social Loafing State in Collaborative Learning with a Hidden Markov Model: A Case on Slack. In Proceedings of the 26th International Conference on World Wide Web Companion (Perth, Australia) (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 149--152. https://doi.org/10.1145/3041021.3054145
[128]
Sabrina Zirkel, Julie A Garcia, and Mary C Murphy. 2015. Experience-sampling research methods and their potential for education research. Educational Researcher, Vol. 44, 1 (2015), 7--16.

Cited By

View all
  • (2024)Modeling Behavior Change for Multi-model At-Risk Students Early Prediction2024 International Symposium on Educational Technology (ISET)10.1109/ISET61814.2024.00020(54-58)Online publication date: 29-Jul-2024
  • (2024)Quantification and prediction of engagementInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10353661:1Online publication date: 1-Feb-2024
  • (2024)ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptionsComputers in Human Behavior: Artificial Humans10.1016/j.chbah.2023.1000272:1(100027)Online publication date: Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 5, Issue CSCW1
CSCW
April 2021
5016 pages
EISSN:2573-0142
DOI:10.1145/3460939
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2021
Published in PACMHCI Volume 5, Issue CSCW1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. dropout prediction
  2. engagement mood
  3. online judge
  4. online learning
  5. problem-based learning (pbl)
  6. question pool website (qp)

Qualifiers

  • Research-article

Funding Sources

  • Research Grants Council of the Hong Kong Special Administrative Region China under General Research Fund (GRF)
  • Research Grants Council of Hong Kong and the 5GEAR and FIT projects from Academy of Finland

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)47
  • Downloads (Last 6 weeks)3
Reflects downloads up to 26 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Modeling Behavior Change for Multi-model At-Risk Students Early Prediction2024 International Symposium on Educational Technology (ISET)10.1109/ISET61814.2024.00020(54-58)Online publication date: 29-Jul-2024
  • (2024)Quantification and prediction of engagementInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10353661:1Online publication date: 1-Feb-2024
  • (2024)ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptionsComputers in Human Behavior: Artificial Humans10.1016/j.chbah.2023.1000272:1(100027)Online publication date: Jan-2024
  • (2023)Envisioning an Inclusive Metaverse: Student Perspectives on Accessible and Empowering Metaverse-Enabled LearningProceedings of the Tenth ACM Conference on Learning @ Scale10.1145/3573051.3596185(346-353)Online publication date: 20-Jul-2023
  • (2023)Accompany Children's Learning for You: An Intelligent Companion Learning SystemComputer Graphics Forum10.1111/cgf.1486242:6Online publication date: 3-Jul-2023
  • (2023)Predicting Dropouts Before Enrollments in MOOCs: An Explainable and Self-Supervised ModelIEEE Transactions on Services Computing10.1109/TSC.2023.331162716:6(4154-4167)Online publication date: Nov-2023
  • (2023)An Evaluation Method for Large Language Models’ Code Generation Capability2023 10th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA59317.2023.00118(831-838)Online publication date: 10-Aug-2023
  • (2023)Machine Learning Models for Predicting Student Dropout—a ReviewProceedings of Eighth International Congress on Information and Communication Technology10.1007/978-981-99-3043-2_83(1003-1014)Online publication date: 1-Sep-2023
  • (2023)Your Favorite Gameplay Speaks Volumes About You: Predicting User Behavior and Hexad TypeHCI in Games10.1007/978-3-031-35979-8_17(210-228)Online publication date: 23-Jul-2023
  • (2022)More Gamification Is Not Always BetterProceedings of the ACM on Human-Computer Interaction10.1145/35555536:CSCW2(1-32)Online publication date: 11-Nov-2022
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media