research-article

Detecting Diligence with Online Behaviors on Intelligent Tutoring Systems

Authors:

Michael Yudelson,

Kenneth R. KoedingerAuthors Info & Claims

L@S '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale

Pages 51 - 59

https://doi.org/10.1145/3051457.3051470

Published: 12 April 2017 Publication History

Abstract

The current study introduces a model for measuring student diligence using online behaviors during intelligent tutoring system use. This model is validated using a full academic year dataset to test its predictive validity against long-term academic outcomes including end-of-year grades and total work completed by the end of the year. The model is additionally validated for robustness to time-sample length as well as data sampling frequency. While the model is shown to be predictive and robust to time-sample length, the results are inconclusive for robustness in data sampling frequency. Implications for research on interventions, and understanding the influence of self-control, motivation, metacognition, and cognition are discussed.

References

[1]

Vincent Aleven, Ido Roll, Bruce M McLaren, and Kenneth R Koedinger. 2010. Automated, Unobtrusive, Action-by-Action Assessment of SelfRegulation During Learning With an Intelligent Tutoring System. Educational Psychologist 45, 4: 224--233.

[2]

Carole Ames and Jennifer Archer. 1988. Achievement goals in the classroom: Students' learning strategies and motivation processes. Journal of Educational Psychology 80, 3: 260--267.

[3]

Ryan Shaun Baker, Albert T Corbett, and Kenneth R Koedinger. 2004. Detecting Student Misuse of Intelligent Tutoring Systems. In Intelligent Tutoring Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 531--540.

[4]

Albert Bandura. 1994. Self-efficacy In VS Ramachaudran (Ed.) Encyclopedia of Human Behavior, 4, 71--81.

[5]

Roy F Baumeister, Brandon Schmeichel, and Kathleen Vohs. 2003. Selfregulation and the executive function of the self. In Social Psychology Handbook of basic Principles (2nd ed.). New York, 197--217.

[6]

Matthew L Bernacki and Steven Ritter. 2013. Hopewell 2011-2012. Dataset 613 in DataShop. Retrieved from https://pslcdatashop.web.cmu.edu/DatasetInfo?data setId=613.

[7]

Matthew L Bernacki, Timothy J Nokes-Malach, and Vincent Aleven. 2013. Fine-Grained Assessment of Motivation over Long Periods of Learning with an Intelligent Tutoring System: Methodology, Advantages, and Preliminary Results. In International Handbook of Metacognition and Learning Technologies. Springer New York, New York, NY, 629--644.

[8]

Jeni L Burnette, Ernest H O'Boyle, Eric M VanEpps, Jeffrey M Pollack, and Eli J Finkel. 2013. Mind-sets matter: A meta-analytic review of implicit theories and self-regulation. Psychological Bulletin 139, 3: 655--701.

[9]

Angela L Duckworth and Laurence Steinberg. 2015. Unpacking Self-Control. Child Development Perspectives 9, 1: 32--37.

[10]

Angela L Duckworth and David Scott Yeager. 2015. Measurement matters assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher 44, 4: 237--251.

[11]

Angela L Duckworth, Tamar Szabó Gendler, and James J Gross. 2016. Situational Strategies for SelfControl. Perspectives on Psychological Science 11, 1: 35--55.

[12]

Carol Dweck. 2000. Self-theories: Their role in motivation, personality, and development. Psychology Press.

[13]

Andrew J Elliot and Kou Murayama. 2008. On the measurement of achievement goals: Critique, illustration, and application. Journal of Educational Psychology 100, 3: 613--628.

[14]

K Anders Ericsson, Ralf T Krampe, and Clemens Tesch-Römer. 1993. The role of deliberate practice in the acquisition of expert performance. Psychological Review 100, 3: 363--406.

[15]

Stephen Fancsali, Matthew L Bernacki, Timothy J Nokes-Malach, Michael Yudelson, and Steven Ritter. 2014. Goal Orientation, Self-Efficacy, and "Online Measures" in Intelligent Tutoring Systems. CogSci.

[16]

Brian M Galla, Benjamin D Plummer, Rachel E White, David Meketon, Sidney K D'Mello, and Angela L Duckworth. 2014. The Academic Diligence Task (ADT): assessing individual differences in effort on tedious but important schoolwork. 39, 4: 314--325.

[17]

Veronika Job, Carol S Dweck, and Gregory M Walton. 2010. Ego Depletion? Is It All in Your Head? Psychological Science 21, 11: 1686--1693.

[18]

Hanan Khalil and Martin Ebner. 2014. MOOCs completion rates and possible methods to improve retention-a literature review. World Conference on Educational Multimedia.

[19]

John S Kinnebrew, Kirk M Loretz, and Gautam Biswas. 2013. A Contextualized, Differential Sequence Mining Method to Derive Students' Learning Behavior Patterns. JEDM - Journal of Educational Data Mining 5, 1: 190--219.

[20]

Kenneth R Koedinger, Albert T Corbett, and Charles Perfetti. 2012. The Knowledge-LearningInstruction Framework: Bridging the SciencePractice Chasm to Enhance Robust Student Learning. Cognitive Science 36, 5: 757--798.

[21]

Kenneth R Koedinger, Jihee Kim, Julianna Zhuxin Jia, Elizabeth A McLaughlin, and Norman L Bier. 2015. Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC. ACM, New York, New York, USA.

Digital Library

[22]

Carolyn MacCann, Angela Lee Duckworth, and Richard D Roberts. 2009. Empirical identification of the major facets of Conscientiousness. Learning and Individual Differences 19, 4: 451--458.

[23]

Gladwell Malcolm. 2008. Outliers: The story of success. New York: Little.

[24]

Walter Mischel, Yuichi Shoda, and Monica L. Rodriguez. 1989. Delay of gratification in children. Science 244, 4907: 933--938.

[25]

National Research Council. 2011. Assessing 21st Century Skills. National Academies Press, Washington, D.C. 26. Paul R Pintrich. 1991. A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ).

[26]

Paul R Pintrich. 2000. An Achievement Goal Theory Perspective on Issues in Motivation Terminology, Theory, and Research. 25, 1: 92--104.

[27]

Steven Ritter, John R Anderson, Kenneth R Koedinger, and Albert Corbett. 2007. Cognitive Tutor: Applied research in mathematics education. Psychonomic Bulletin & Review 14, 2: 249--255.

[28]

Nicole Shechtman, Angela H DeBarger, Carolyn Dornsife, and Soren Rosier. 2013. Promoting grit, tenacity, and perseverance: Critical factors for success in the 21st century. Washington.

[29]

June P Tangney, Roy F Baumeister, and Angie Luzio Boone. 2004. High Self-Control Predicts Good Adjustment, Less Pathology, Better Grades, and Interpersonal Success. Journal of personality 72, 2: 271--324.

[30]

Eli Tsukayama, Angela Lee Duckworth, and Betty Kim. 2013. Domain-specific impulsivity in schoolage children. Developmental Science 16, 6: 879--893.

[31]

Phillip H Winne, John C Nesbit, Vive Kumar, et al. 2006. Supporting self-regulated learning with gStudy software: The Learning Kit Project. Technology.

[32]

Barry J Zimmerman. 2008. Investigating SelfRegulation and Motivation: Historical Background, Methodological Developments, and Future Prospects. American Educational Research Journal 45, 1: 166--183.

Cited By

Hasnawati EFitriyanti EHasan SIsmail S(2024)Developing Indonesian Vocabulary Through the Application of the Mind Mapping Method in ChildrenJournal of Basic Education Research10.37251/jber.v5i1.8275:1(34-39)Online publication date: 31-Jan-2024
https://doi.org/10.37251/jber.v5i1.827
Tellakat MBoyd RPennebaker J(2019)How do online learners study? The psychometrics of students’ clicking patterns in online coursesPLOS ONE10.1371/journal.pone.021386314:3(e0213863)Online publication date: 25-Mar-2019
https://doi.org/10.1371/journal.pone.0213863
Motz BQuick JSchroeder NZook JGunkel MHsiao SCunningham JMcCarthy KLynch GBrooks CFerguson RHoppe U(2019)The validity and utility of activity logs as a measure of student engagementProceedings of the 9th International Conference on Learning Analytics & Knowledge10.1145/3303772.3303789(300-309)Online publication date: 4-Mar-2019
https://dl.acm.org/doi/10.1145/3303772.3303789
Show More Cited By

Index Terms

Detecting Diligence with Online Behaviors on Intelligent Tutoring Systems
1. Applied computing
  1. Education
    1. E-learning
    2. Interactive learning environments

Recommendations

Educational Game and Intelligent Tutoring System: A Classroom Study and Comparative Design Analysis

Educational games and intelligent tutoring systems (ITS) both support learning by doing, although often in different ways. The current classroom experiment compared a popular commercial game for equation solving, DragonBox and a research-based ITS, ...
Are There Benefits of Using Multiple Pedagogical Agents to Support and Foster Self-Regulated Learning in an Intelligent Tutoring System?
ITS 2016: Proceedings of the 13th International Conference on Intelligent Tutoring Systems - Volume 9684

This study examined the proportional learning gains attained by 165 college students as they learned about the human circulatory system over two sessions with the intelligent tutoring system, MetaTutor. Results indicated that learners in the prompt and ...
Motivation matters

Pedagogical agent scaffolding increased learners' use of self-regulatory strategies.Pedagogical agent scaffolding alone did not increase learners' performance.Achievement goal motivation moderated the effect of agents on performance.Performance was ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

L@S '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale

April 2017

352 pages

ISBN:9781450344500

DOI:10.1145/3051457

General Chair:
Claudia Urrea
Massachusetts Institute of Technology, USA
,
Program Chairs:
Justin Reich
Massachusetts Institute of Technology, USA
,
Candace Thille
Stanford University, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 April 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Institute of Education Sciences

Conference

L@S 2017

Sponsor:

ACM

L@S 2017: Fourth (2017) ACM Conference on Learning @ Scale

April 20 - 21, 2017

Massachusetts, Cambridge, USA

Acceptance Rates

L@S '17 Paper Acceptance Rate 14 of 105 submissions, 13%;

Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
515
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)11

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hasnawati EFitriyanti EHasan SIsmail S(2024)Developing Indonesian Vocabulary Through the Application of the Mind Mapping Method in ChildrenJournal of Basic Education Research10.37251/jber.v5i1.8275:1(34-39)Online publication date: 31-Jan-2024
https://doi.org/10.37251/jber.v5i1.827
Tellakat MBoyd RPennebaker J(2019)How do online learners study? The psychometrics of students’ clicking patterns in online coursesPLOS ONE10.1371/journal.pone.021386314:3(e0213863)Online publication date: 25-Mar-2019
https://doi.org/10.1371/journal.pone.0213863
Motz BQuick JSchroeder NZook JGunkel MHsiao SCunningham JMcCarthy KLynch GBrooks CFerguson RHoppe U(2019)The validity and utility of activity logs as a measure of student engagementProceedings of the 9th International Conference on Learning Analytics & Knowledge10.1145/3303772.3303789(300-309)Online publication date: 4-Mar-2019
https://dl.acm.org/doi/10.1145/3303772.3303789
Kross SGuo PLuckin RKlemmer SKoedinger K(2018)Students, systems, and interactionsProceedings of the Fifth Annual ACM Conference on Learning at Scale10.1145/3231644.3231662(1-10)Online publication date: 26-Jun-2018
https://dl.acm.org/doi/10.1145/3231644.3231662

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents