Article

Detecting carelessness through contextual estimation of slip probabilities among students using an intelligent tutor for mathematics

Authors:

Maria Ofelia Clarissa Z. San Pedro,

Ryan S. J. D. Baker,

Ma. Mercedes T. RodrigoAuthors Info & Claims

AIED'11: Proceedings of the 15th international conference on Artificial intelligence in education

Pages 304 - 311

Published: 28 June 2011 Publication History

Abstract

A student is said to have committed a careless error when a student's answer is wrong despite the fact that he or she knows the answer (Clements, 1982). In this paper, educational data mining techniques are used to analyze log files produced by a cognitive tutor for Scatterplots to derive a model and detector for carelessness. Bayesian Knowledge Tracing and its variant, the Contextual-Slip-and-Guess Estimation, are used to model and predict carelessness behavior in the Scatterplot Tutor. The study examines as well the robustness of this detector to a major difference in the tutor's interface, namely the presence or absence of an embodied conversational agent, as well as robustness to data from a different school setting (USA versus Philippines).

References

[1]

Arroyo, I., Cooper, D., Burleson, W., Woolf, B.P., Muldner, K., Christopherson, R.: Emotion Sensors Go To School. In: Proceedings of the International Conference on Artificial Intelligence in Education, pp. 17-24 (2009).

Digital Library

[2]

Baker, R.S.J.d.: Modeling and Understanding Students' Off-Task Behavior in Intelligent Tutoring Systems. In: Proceedings of ACM CHI 2007: Computer-Human Interaction, pp. 1059-1068 (2007).

Digital Library

[3]

Baker, R.S.J.d., Corbett, A.T., Aleven, V.: Improving Contextual Models of Guessing and Slipping with a Truncated Training Set. In: Proceedings of the 1st International Conference on Educational Data Mining, pp. 67-76 (2008).

[4]

Baker, R.S.J.d., Corbett, A.T., Aleven, V.: More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 406-415. Springer, Heidelberg (2008).

Digital Library

[5]

Baker, R.S.J.d., Corbett, A.T., Gowda, S.M., Wagner, A.Z., MacLaren, B.M., Kauffman, L.R., Mitchell, A.P., Giguere, S.: Contextual Slip and Prediction of Student Performance After Use of an Intelligent Tutor. In: Proceedings of the 18th Annual Conference on User Modeling, Adaptation, and Personalization (2008).

Digital Library

[6]

Baker, R.S.J.d., Corbett, A.T., Koedinger, K.R., Evenson, S.E., Roll, I., Wagner, A.Z., Naim, M., Raspat, J., Baker, D.J., Beck, J.: Adapting to When Students Game an Intelligent Tutoring System. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 392-401. Springer, Heidelberg (2006).

Digital Library

[7]

Baker, R.S.J.d., D'Mello, S.K., Rodrigo, M.M.T., Graesser, A.C.: Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments. International Journal of Human-Computer Studies 68(4), 223-241 (2010).

Digital Library

[8]

Baker, R.S.J.d., Gowda, S.M.: An Analysis of the Differences in the Frequency of Students' Disengagement in Urban, Rural, and Suburban High Schools. In: Proceedings of the 3rd International Conference on Educational Data Mining, pp. 11-20 (2010).

[9]

Clements, M.A.: Analysing Children's Errors on Written Mathematical Tasks. Educational Studies in Mathematics, 1-21 (1982).

[10]

Clements, M.A.: Careless Errors Made by Sixth-grade Children on Written Mathematical Tasks. Journal for Research in Mathematics Education 13(2), 136-144 (1982).

[11]

Corbett, A.T., Anderson, J.R.: Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User-Adapted Interaction 4, 253-278 (1995).

[12]

Craighead, W.E.: The Concise Corsini Encyclopedia of Psychology and Behavioral Science, 3rd edn. (2004).

[13]

Dix, A., Finlay, J., Abowd, G., Beale, R.: Human-Computer Interaction. Prentice Hall, Englewood Cliffs (1993).

Digital Library

[14]

Fogarty, J., Baker, R.S.J.d., Hudson, S.E.: Case Studies in the Use of ROC Curve Analysis for Sensor-based Estimates in Human Computer Interaction. In: Proceedings of Graphics Interface, pp. 129-136 (2005).

Digital Library

[15]

Levitin, D.J. (ed.): Foundations of Cognitive Psychology: Core Readings. MIT Press, Cambridge (2002).

[16]

Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.: Intelligent Tutoring Goes to School in the Big City. International Journal of Artificial Intelligence in Education 8, 30-43 (1997).

[17]

McQuiggan, S.W., Lee, S., Lester, J.C.: Early Prediction of Student Frustration. In: Paiva, A., Prada, R., Picard, R.W. (eds.) Affective Computing and Intelligent Interaction, pp. 698-709 (2007).

Digital Library

[18]

Rafferty, A.E.: Bayesian Model Selection in Social Research. Sociological Methodology 25, 111-163 (2003).

[19]

Rosenthal, R., Rosnow, R.L.: Essentials of Behavioural Research: Methods and Data Analysis. McGraw-Hill Humanities, New York (2008).

Cited By

Holstein KMcLaren BAleven VWise AWinne PLynch GOchoa XMolenaar IDawson SHatala M(2017)SPACLEProceedings of the Seventh International Learning Analytics & Knowledge Conference10.1145/3027385.3027450(358-367)Online publication date: 13-Mar-2017
https://dl.acm.org/doi/10.1145/3027385.3027450
Miller WBaker RLabrum MPetsche KLiu YWagner ABaron JLynch GMaziarz NBlikstein PMerceron ASiemens G(2015)Automated detection of proactive remediation by teachers in reasoning mind classroomsProceedings of the Fifth International Conference on Learning Analytics And Knowledge10.1145/2723576.2723607(290-294)Online publication date: 16-Mar-2015
https://dl.acm.org/doi/10.1145/2723576.2723607
Sao Pedro MGobert JBetts C(2014)Towards Scalable Assessment of Performance-Based Skills12th International Conference on Intelligent Tutoring Systems - Volume 847410.1007/978-3-319-07221-0_75(591-600)Online publication date: 5-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07221-0_75
Show More Cited By

Recommendations

A New Paradigm for Intelligent Tutoring Systems: Example-Tracing Tutors

The Cognitive Tutor Authoring Tools (CTAT) support creation of a novel type of tutors called example-tracing tutors. Unlike other types of ITSs (e.g., model-tracing tutors, constraint-based tutors), exampletracing tutors evaluate student behavior by ...
The relationship between carelessness and affect in a cognitive tutor
ACII'11: Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I

We study the relationship between student carelessness and affect among high-school students using a Cognitive Tutor for Scatterplots, using a machine-learned detector of carelessness and field observations of student affect. In line with previous ...
Contextual slip and prediction of student performance after use of an intelligent tutor
UMAP'10: Proceedings of the 18th international conference on User Modeling, Adaptation, and Personalization

Intelligent tutoring systems that utilize Bayesian Knowledge Tracing have achieved the ability to accurately predict student performance not only within the intelligent tutoring system, but on paper post-tests outside of the system Recent work has ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

AIED'11: Proceedings of the 15th international conference on Artificial intelligence in education

June 2011

638 pages

ISBN:9783642218682

Editors:
Gautam Biswas
Vanderbilt University, EECS Department, Nashville, TN
,
Susan Bull
The University of Birmingham, Electronic, Electrical and Computer Engineering, UK
,
Judy Kay
University of Sydney, School of Information Technologies, Australia
,
Antonija Mitrovic
University of Canterbury, College of Engineering, Department of Computer Science and Software Engineering, New Zealand

Sponsors

The University of Canterbury
The University of Auckland, New Zealand
The University of Memphis

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 28 June 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Holstein KMcLaren BAleven VWise AWinne PLynch GOchoa XMolenaar IDawson SHatala M(2017)SPACLEProceedings of the Seventh International Learning Analytics & Knowledge Conference10.1145/3027385.3027450(358-367)Online publication date: 13-Mar-2017
https://dl.acm.org/doi/10.1145/3027385.3027450
Miller WBaker RLabrum MPetsche KLiu YWagner ABaron JLynch GMaziarz NBlikstein PMerceron ASiemens G(2015)Automated detection of proactive remediation by teachers in reasoning mind classroomsProceedings of the Fifth International Conference on Learning Analytics And Knowledge10.1145/2723576.2723607(290-294)Online publication date: 16-Mar-2015
https://dl.acm.org/doi/10.1145/2723576.2723607
Sao Pedro MGobert JBetts C(2014)Towards Scalable Assessment of Performance-Based Skills12th International Conference on Intelligent Tutoring Systems - Volume 847410.1007/978-3-319-07221-0_75(591-600)Online publication date: 5-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07221-0_75
Pardos ZBaker RSan Pedro MGowda SGowda S(2013)Affective states and state testsProceedings of the Third International Conference on Learning Analytics and Knowledge10.1145/2460296.2460320(117-124)Online publication date: 8-Apr-2013
https://dl.acm.org/doi/10.1145/2460296.2460320
Pedro MRodrigo MBaker R(2011)The relationship between carelessness and affect in a cognitive tutorProceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I10.5555/2062780.2062816(306-315)Online publication date: 9-Oct-2011
https://dl.acm.org/doi/10.5555/2062780.2062816

View Options

View options

Figures

Tables

Media

View Table of Conten