Article

Educational Software Features that Encourage and Discourage “Gaming the System”

Authors:

Ryan S. J. d. Baker,

Adriana M. J. B. De Carvalho,

Vincent Aleven,

Albert T. Corbett,

Kenneth R. KoedingerAuthors Info & Claims

Proceedings of the 2009 conference on Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling

Pages 475 - 482

Published: 20 July 2009 Publication History

Abstract

Gaming the system, attempting to succeed in an interactive learning environment by exploiting properties of the system rather than by learning the material (for example, by systematically guessing or abusing hints), is prevalent across many types of educational software. Past research on why students choose to game has focused on student individual differences. Many student individual differences, including attitudes towards mathematics, have been shown to be associated with gaming, but generally with low correlation. In this paper, we investigate how individual differences between learning environments can increase or decrease the probability of gaming. We enumerate ways intelligent tutor lessons vary from each other, and use data mining to discover hypotheses about how differences in software design and content influence the choice to game the system. We discover a set of tutor features that explain 56% of the variance in gaming, over five times the degree of variance explained in any prior study of student individual differences and gaming. These results provide an important step towards developing prescriptions for designing intelligent tutor software that students game significantly less.

References

[1]

Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., Barto, A., Mahadevan, S., Woolf. B.P. Repairing Disengagement with Non-Invasive Interventions. Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED-2007), 195-202.

[2]

Arroyo, I., & Woolf, B. Inferring learning and attitudes from a Bayesian network of log file data. Proceedings of the 12th International Conference on Artificial Intelligence in Education (2005), 33-40.

[3]

Baker, R.S.J.d., Corbett, A.T., Koedinger, K.R., Evenson, S.E., Roll, I., Wagner, A.Z., Naim, M., Raspat, J., Baker, D.J., Beck, J. Adapting to When Students Game an Intelligent Tutoring System. Proceedings of the 8th International Conference on Intelligent Tutoring Systems (2006), 392-401.

[4]

Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z. Off-Task Behavior in the Cognitive Tutor Classroom: When Students "Game The System". Proc. ACM CHI 2004, 383-390.

[5]

Baker, R.S.J.d. Is Gaming the System State-or-Trait? Educational Data Mining Through the Multi-Contextual Application of a Validated Behavioral Model. Proc. of the Workshop on Data Mining for User Modeling at the 11th International Conference on User Modeling 2007, 76-80.

[6]

Baker, R.S.J.d., Corbett, A.T., Wagner, A.Z. Human Classification of Low-Fidelity Replays of Student Actions. Proc. Edu. Data Mining Workshop, Int'l Conf. on Intelligent Tutoring Systems (2006), 29-36.

[7]

Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., Koedinger, K. Why Students Engage in "Gaming the System" Behavior in Interactive Learning Environments. Journal of Interactive Learning Research 19(2) (2008), 185-224.

[8]

Beal, C.R., Qu, L., Lee, H. Mathematics motivation and achievement as predictors of high school students' guessing and help-seeking with instructional software. J. Comp. Assisted Learning, in press.

[9]

Beck, J.E. Using Learning Decomposition to Analyze Student Fluency Development. Proc. Edu. Data Mining Workshop, 8th Int'l Conf. on Intelligent Tutoring Systems (2006), 21-28.

[10]

Corbett, A.T., & Anderson, J.R. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4 (1995), 253-278.

[11]

Csikszentmihalyi, M., Schneider, B. Becoming Adult: How Teenagers Prepare for the World of Work. New York: Basic Books, 2000.

[12]

Kincaid, J. P., Fishburne, R. P., Rogers, R. L., Chissom, B. S. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8-75, Naval Technical Training, 1975.

[13]

Koedinger, K., Cunningham, K., Skogsholm A., Leber, B. An open repository and analysis tools for fine-grained, longitudinal learner data. Proc. 1st Int'l Conf. on Edu. Data Mining (2008), 157-166.

[14]

Murray, R.C., & vanLehn, K. Effects of dissuading unnecessary help requests while providing proactive help. Proc. of the 12th Int'l Conf. on Artificial Intelligence in Education (2005), 887-889.

[15]

Rodrigo, M.M.T., Baker, R.S.J.d., Lagud, M.C.V., Lim, S.A.L., Macapanpan, A.F., Pascua, S.A.M.S., Santillano, J.Q., Sevilla, L.R.S., Sugay, J.O., Tep, S., Viehland, N.J.B. Affect and Usage Choices in Simulation Problem Solving Environments. Proceedings of the 13th International Conference on Artificial Intelligence in Education (2007), 145-152.

[16]

Roll, I., Aleven, V., McLaren, B. M., Koedinger, K. R. Can help seeking be tutored? Searching for the secret sauce of metacognitive tutoring. Proceedings of the 13th International Conference on Artificial Intelligence in Education (2007).

[17]

Walonoski, J.A., & Heffernan, N.T. Prevention of off-task gaming behavior in intelligent tutoring systems. Proc. of the 8th International Conference on Intelligent Tutoring Systems (2006), 722-724.

Cited By

Slater SBaker RAlmeda MBowers AHeffernan NWise AWinne PLynch GOchoa XMolenaar IDawson SHatala M(2017)Using correlational topic modeling for automated topic identification in intelligent tutoring systemsProceedings of the Seventh International Learning Analytics & Knowledge Conference10.1145/3027385.3027438(393-397)Online publication date: 13-Mar-2017
https://dl.acm.org/doi/10.1145/3027385.3027438
Alexandron GRuiprez-Valiente JChen ZMuoz-Merino PPritchard D(2017)Copying@ScaleComputers & Education10.1016/j.compedu.2017.01.015108:C(96-114)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1016/j.compedu.2017.01.015
Easterday MJo I(2014)Replay Penalties in Cognitive Games12th International Conference on Intelligent Tutoring Systems - Volume 847410.1007/978-3-319-07221-0_48(388-397)Online publication date: 5-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07221-0_48
Show More Cited By

Educational Software Features that Encourage and Discourage “Gaming the System”
1. Applied computing

Recommendations

Exploring Gamification to Prevent Gaming the System and Help Refusal in Tutoring Systems
Lifelong Technology-Enhanced Learning
Abstract
Intelligent Tutoring Systems (ITSs) have shown to be almost as effective as one-to-one tutoring. Nonetheless, the students’ improper use of the ITS help system and its intelligent assistance, i. e. gaming the system or help refusal, can impair ...
Detecting gaming the system in constraint-based tutors
UMAP'10: Proceedings of the 18th international conference on User Modeling, Adaptation, and Personalization

Recently, detectors of gaming the system have been developed for several intelligent tutoring systems where the problem-solving process is reified, and gaming consists of systematic guessing and help abuse Constraint-based tutors differ from the tutors ...
Confrustion and Gaming While Learning with Erroneous Examples in a Decimals Game
Artificial Intelligence in Education
Abstract
Prior studies have explored the potential of erroneous examples in helping students learn more effectively by correcting errors in solutions to decimal problems. One recent study found that while students experience more confusion and frustration (...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

Proceedings of the 2009 conference on Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling

July 2009

832 pages

ISBN:9781607500285

Editors:
Vania Dimitrova
University of Leeds, UK
,
Riichiro Mizoguchi
Osaka University, Japan
,
Benedict du Boulay
University of Sussex, UK
,
Art Graesser
University of Memphis, USA

Publisher

IOS Press

Netherlands

Publication History

Published: 20 July 2009

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Slater SBaker RAlmeda MBowers AHeffernan NWise AWinne PLynch GOchoa XMolenaar IDawson SHatala M(2017)Using correlational topic modeling for automated topic identification in intelligent tutoring systemsProceedings of the Seventh International Learning Analytics & Knowledge Conference10.1145/3027385.3027438(393-397)Online publication date: 13-Mar-2017
https://dl.acm.org/doi/10.1145/3027385.3027438
Alexandron GRuiprez-Valiente JChen ZMuoz-Merino PPritchard D(2017)Copying@ScaleComputers & Education10.1016/j.compedu.2017.01.015108:C(96-114)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1016/j.compedu.2017.01.015
Easterday MJo I(2014)Replay Penalties in Cognitive Games12th International Conference on Intelligent Tutoring Systems - Volume 847410.1007/978-3-319-07221-0_48(388-397)Online publication date: 5-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07221-0_48
Wixon MBaker RGobert JOcumpaugh JBachmann M(2012)WTF? detecting students who are conducting inquiry without thinking fastidiouslyProceedings of the 20th international conference on User Modeling, Adaptation, and Personalization10.1007/978-3-642-31454-4_24(286-296)Online publication date: 16-Jul-2012
https://dl.acm.org/doi/10.1007/978-3-642-31454-4_24
Gonzalez-Sanchez JChavez-Echeagaray MVanlehn KBurleson WHvatum L(2011)From behavioral description to a pattern-based model for intelligent tutoring systemsProceedings of the 18th Conference on Pattern Languages of Programs10.1145/2578903.2579164(1-12)Online publication date: 21-Oct-2011
https://dl.acm.org/doi/10.1145/2578903.2579164
Louis AHiggins DTetreault JBurstein JLeacock C(2010)Off-topic essay detection using short prompt textsProceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications10.5555/1866795.1866808(92-95)Online publication date: 5-Jun-2010
https://dl.acm.org/doi/10.5555/1866795.1866808
Girard SJohnson HCastellano GKarpouzis KMartin JMorency LPeters CRiek L(2010)Designing affective computing learning companions with teachers as design partners.Proceedings of the 3rd international workshop on Affective interaction in natural environments10.1145/1877826.1877840(49-54)Online publication date: 29-Oct-2010
https://dl.acm.org/doi/10.1145/1877826.1877840
Baker RMitrović AMathews M(2010)Detecting gaming the system in constraint-based tutorsProceedings of the 18th international conference on User Modeling, Adaptation, and Personalization10.1007/978-3-642-13470-8_25(267-278)Online publication date: 20-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13470-8_25
Hastings PArnott-Hill EAllbritton D(2010)Squeezing out gaming behavior in a dialog-based ITSProceedings of the 10th international conference on Intelligent Tutoring Systems - Volume Part I10.1007/978-3-642-13388-6_25(204-213)Online publication date: 14-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13388-6_25
Muldner KBurleson WVan de Sande BVanLehn K(2010)An analysis of gaming behaviors in an intelligent tutoring systemProceedings of the 10th international conference on Intelligent Tutoring Systems - Volume Part I10.1007/978-3-642-13388-6_23(184-193)Online publication date: 14-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13388-6_23

View Options

View options

Figures

Tables

Media

View Table of Conten