Discovering Players’ Problem-Solving Behavioral Characteristics in a Puzzle Game through Sequence Mining
Pages 498 - 506
Abstract
Digital games offer promising platforms for assessing student higher-order competencies such as problem-solving. However, processing and analyzing the large volume of interaction log data generated in these platforms to uncover meaningful behavioral patterns remain a complex research challenge. In this study, we employ sequence mining and clustering techniques to examine students’ log data in an interactive puzzle game that requires player to change rules to win the game. Our goal is to identify behavioral characteristics associated with the problem-solving practices adopted by individual students. The findings indicate that the most effective problem solvers made fewer rule changes and took longer time to make those changes across both an introductory and a more advanced level of the game. Conversely, rapid rule change actions were linked to ineffective problem-solving. This research underscores the potential of sequence mining and cluster analysis as generalizable methods for understanding student higher-order competencies through log data in digital gaming and learning environments. It also suggests future directions on how to provide just-in-time, in-game feedback to enhance student problem-solving competences.
References
[1]
Andrew Abbott and Angela Tsay. 2000. Sequence analysis and optimal matching methods in sociology: Review and prospect. Sociological methods & research 29, 1 (2000), 3–33.
[2]
Saleema Amershi, Cristina Conati, 2009. Combining unsupervised and supervised classification to build user models for exploratory learning environments. Journal of educational data mining 1, 1 (2009), 18–71.
[3]
Dylan A Arena and Daniel L Schwartz. 2014. Experience and explanation: Using videogames to prepare students for formal instruction in statistics. Journal of Science Education and Technology 23 (2014), 538–548.
[4]
Ryan Shaun Baker, Albert T Corbett, Kenneth R Koedinger, and Angela Z Wagner. 2004. Off-task behavior in the cognitive tutor classroom: When students" game the system". In Proceedings of the SIGCHI conference on Human factors in computing systems. 383–390.
[5]
Elizabeth A Boyle, Thomas M Connolly, Thomas Hainey, and James M Boyle. 2012. Engagement in digital entertainment games: A systematic review. Computers in human behavior 28, 3 (2012), 771–780.
[6]
Elizabeth A Boyle, Thomas Hainey, Thomas M Connolly, Grant Gray, Jeffrey Earp, Michela Ott, Theodore Lim, Manuel Ninaus, Claudia Ribeiro, and João Pereira. 2016. An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education 94 (2016), 178–192.
[7]
Engin Bumbacher, Shima Salehi, Carl Wieman, and Paulo Blikstein. 2018. Tools for science inquiry learning: Tool affordances, experimentation strategies, and conceptual understanding. Journal of Science Education and Technology 27 (2018), 215–235.
[8]
EW Burkholder, JK Miles, TJ Layden, KD Wang, AV Fritz, and CE Wieman. 2020. Template for teaching and assessment of problem solving in introductory physics. Physical Review Physics Education Research 16, 1 (2020), 010123.
[9]
William G Chase and Herbert A Simon. 1973. Perception in chess. Cognitive psychology 4, 1 (1973), 55–81.
[10]
Mihaly Csikszentmihalyi and Isabella Selega Csikszentmihalyi. 1992. Optimal experience: Psychological studies of flow in consciousness. Cambridge university press.
[11]
David DeLiema, Megan Goeke, Basel Hussein, Jesslyn Valerie, Craig Anderson, Keisha Varma, Bodong Chen, Shima Salehi, and Matthew Bernacki. 2022. Playful Learning Following Deviations: A Mixture of Tinkering, Causal Explanations, and Revision Rationales. In Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, pp. 1421-1424. International Society of the Learning Sciences.
[12]
Susan Dumais, Robin Jeffries, Daniel M Russell, Diane Tang, and Jaime Teevan. 2014. Understanding user behavior through log data and analysis. Ways of Knowing in HCI (2014), 349–372.
[13]
Beate Eichmann, Samuel Greiff, Johannes Naumann, Liene Brandhuber, and Frank Goldhammer. 2020. Exploring behavioural patterns during complex problem-solving. Journal of Computer Assisted Learning 36, 6 (2020), 933–956.
[14]
Philippe Fournier-Viger, Jerry Chun-Wei Lin, Rage Uday Kiran, Yun Sing Koh, and Rincy Thomas. 2017. A survey of sequential pattern mining. Data Science and Pattern Recognition 1, 1 (2017), 54–77.
[15]
Peter A Frensch and Joachim Funke. 2014. Definitions, traditions, and a general framework for understanding complex problem solving. In Complex problem solving. Psychology Press, 3–25.
[16]
Alexis Gabadinho, Gilbert Ritschard, Nicolas S Müller, and Matthias Studer. 2011. Analyzing and visualizing state sequences in R with TraMineR. Journal of statistical software 40 (2011), 1–37.
[17]
Janice D Gobert, Michael Sao Pedro, Juelaila Raziuddin, and Ryan S Baker. 2013. From log files to assessment metrics: Measuring students’ science inquiry skills using educational data mining. Journal of the Learning Sciences 22, 4 (2013), 521–563.
[18]
A-M Hoskinson, Marcos D Caballero, and Jennifer K Knight. 2013. How can we improve problem solving in undergraduate biology? Applying lessons from 30 years of physics education research. CBE—Life Sciences Education 12, 2 (2013), 153–161.
[19]
David H Jonassen. 1997. Instructional design models for well-structured and III-structured problem-solving learning outcomes. Educational technology research and development 45, 1 (1997), 65–94.
[20]
Alboukadel Kassambara. 2017. Practical guide to cluster analysis in R: Unsupervised machine learning. Vol. 1. Sthda.
[21]
Deirdre Kerr and Gregory KWK Chung. 2012. Identifying key features of student performance in educational video games and simulations through cluster analysis.Journal of Educational Data Mining 4, 1 (2012), 144–182.
[22]
John S Kinnebrew, Kirk M Loretz, and Gautam Biswas. 2013. A contextualized, differential sequence mining method to derive students’ learning behavior patterns.Journal of Educational Data Mining 5, 1 (2013), 190–219.
[23]
Tongxi Liu and Maya Israel. 2022. Uncovering students’ problem-solving processes in game-based learning environments. Computers & Education 182 (2022), 104462.
[24]
Richard E Mayer. 1992. Thinking, problem solving, cognition. WH Freeman/Times Books/Henry Holt & Co.
[25]
Richard E Mayer and Merlin C Wittrock. 1996. Problem-solving transfer. Handbook of educational psychology (1996), 47–62.
[26]
Fionn Murtagh and Pierre Legendre. 2014. Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion?Journal of classification 31 (2014), 274–295.
[27]
Allen Newell, Herbert Alexander Simon, 1972. Human problem solving. Vol. 104. Prentice-hall Englewood Cliffs, NJ.
[28]
Frank Nielsen and Frank Nielsen. 2016. Hierarchical clustering. Introduction to HPC with MPI for Data Science (2016), 195–211.
[29]
Dilhan Perera, Judy Kay, Irena Koprinska, Kalina Yacef, and Osmar R Zaïane. 2008. Clustering and sequential pattern mining of online collaborative learning data. IEEE Transactions on knowledge and Data Engineering 21, 6 (2008), 759–772.
[30]
Sarah Perez, Jonathan Massey-Allard, Deborah Butler, Joss Ives, Doug Bonn, Nikki Yee, and Ido Roll. 2017. Identifying productive inquiry in virtual labs using sequence mining. In Artificial Intelligence in Education: 18th International Conference, AIED 2017, Wuhan, China, June 28–July 1, 2017, Proceedings 18. Springer, 287–298.
[31]
Argenta M Price, Candice J Kim, Eric W Burkholder, Amy V Fritz, and Carl E Wieman. 2021. A detailed characterization of the expert problem-solving process in science and engineering: Guidance for teaching and assessment. CBE—Life Sciences Education 20, 3 (2021), ar43.
[32]
Frederick Reif and Joan I Heller. 1982. Knowledge structure and problem solving in physics. Educational psychologist 17, 2 (1982), 102–127.
[33]
S Ian Robertson. 2016. Problem solving: Perspectives from cognition and neuroscience. Psychology Press.
[34]
Shima Salehi. 2018. Improving problem-solving through reflection. Stanford University.
[35]
Ketan Rajshekhar Shahapure and Charles Nicholas. 2020. Cluster quality analysis using silhouette score. In 2020 IEEE 7th international conference on data science and advanced analytics (DSAA). IEEE, 747–748.
[36]
Valerie J Shute, Lubin Wang, Samuel Greiff, Weinan Zhao, and Gregory Moore. 2016. Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior 63 (2016), 106–117.
[37]
Michelle Taub, Roger Azevedo, Amanda E Bradbury, Garrett C Millar, and James Lester. 2018. Using sequence mining to reveal the efficiency in scientific reasoning during STEM learning with a game-based learning environment. Learning and instruction 54 (2018), 93–103.
[38]
Karen D Wang, Jade Maï Cock, Tanja Käser, and Engin Bumbacher. 2023. A systematic review of empirical studies using log data from open-ended learning environments to measure science and engineering practices. British Journal of Educational Technology 54, 1 (2023), 192–221.
[39]
Karen D Wang and Carl Wieman. 2022. Applying Sequence Mining to Explore Students’ Problem-Solving Practices Using an Interactive Simulation-Based Task. In Proceedings of the 16th International Conference of the Learning Sciences-ICLS 2022, pp. 433-439. International Society of the Learning Sciences.
Index Terms
- Discovering Players’ Problem-Solving Behavioral Characteristics in a Puzzle Game through Sequence Mining
Recommendations
The impact of learner metacognition and goal orientation on problem-solving in a serious game environment
AbstractTo understand the impact of learner metacognition and goal orientation on problem-solving, this study investigated 159 undergraduate students’ metacognition, goal orientations, and their problem-solving performances and processes in a ...
Highlights- Computer log data was used to analyze leaner problem-solving process.
- ...
Evaluating multiple aspects of a digital educational problem-solving-based adventure game
This study aims to evaluate multiple aspects of a problem-solving-based educational adventure game, Boom Room(C). The learning effectiveness, game acceptance, and flow experience of the game were empirically investigated. The game was designed and ...
A novel Boolean algebraic framework for association and pattern mining
Data mining has been defined as the non- trivial extraction of implicit, previously unknown and potentially useful information from data. Association mining and sequential mining analysis are considered as crucial components of strategic control over a ...
Comments
Information & Contributors
Information
Published In
March 2024
962 pages
ISBN:9798400716188
DOI:10.1145/3636555
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 March 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Carinal Fund
Conference
LAK '24
Acceptance Rates
Overall Acceptance Rate 236 of 782 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 304Total Downloads
- Downloads (Last 12 months)304
- Downloads (Last 6 weeks)67
Reflects downloads up to 17 Oct 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in