Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3331184.3331195acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Knowledge Tracing with Sequential Key-Value Memory Networks

Published: 18 July 2019 Publication History

Abstract

Can machines trace human knowledge like humans? Knowledge tracing (KT) is a fundamental task in a wide range of applications in education, such as massive open online courses (MOOCs), intelligent tutoring systems, educational games, and learning management systems. It models dynamics in a student's knowledge states in relation to different learning concepts through their interactions with learning activities. Recently, several attempts have been made to use deep learning models for tackling the KT problem. Although these deep learning models have shown promising results, they have limitations: either lack the ability to go deeper to trace how specific concepts in a knowledge state are mastered by a student, or fail to capture long-term dependencies in an exercise sequence. In this paper, we address these limitations by proposing a novel deep learning model for knowledge tracing, namely Sequential Key-Value Memory Networks (SKVMN). This model unifies the strengths of recurrent modelling capacity and memory capacity of the existing deep learning KT models for modelling student learning. We have extensively evaluated our proposed model on five benchmark datasets. The experimental results show that (1) SKVMN outperforms the state-of-the-art KT models on all datasets, (2) SKVMN can better discover the correlation between latent concepts and questions, and (3) SKVMN can trace the knowledge state of students dynamics, and a leverage sequential dependencies in an exercise sequence for improved predication accuracy.

Supplementary Material

MP4 File (cite3-13h30-d1.mp4)

References

[1]
Ryan S. Baker, Albert T. Corbett, and Vincent Aleven. 2008. More Accurate Student Modeling Through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems (ITS). Berlin, Heidelberg, 406--415.
[2]
Yoshua Bengio. 2012. Practical recommendations for gradient-based training of deep architectures. Neural networks: Tricks of the trade: Second Edition. Berlin, Heidelberg, 437--478.
[3]
Léon Bottou. 2012. Stochastic gradient descent tricks. Neural networks: Tricks of the trade: Second Edition. Berlin, Heidelberg, 421--436.
[4]
Víctor Campos, Brendan Jou, Xavier Giró i Nieto, Jordi Torres, and Shih-Fu Chang. 2018. Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks. In 6th International Conference on Learning Representations, (ICLR), Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.
[5]
Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. 2015. Modeling Exercise Relationships in E-Learning: A Unified Approach. In Proceedings of the 8th International Conference on Educational Data Mining, (EDM), Madrid, Spain, June 26-29, 2015. 532--535.
[6]
Albert T. Corbett and John R. Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, Vol. 4, 4 (01 Dec 1994), 253--278.
[7]
Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. 2015. Long-Term Recurrent Convolutional Networks for Visual Recognition and Description. In The IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), Boston, MA, USA, June 7-12, 2015. 2625--2634.
[8]
Charles R. Gallistel and Adam Philip King. 2011. Memory and the computational brain: Why cognitive science will transform neuroscience. Vol. 6. John Wiley & Sons.
[9]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). Society for Artificial Intelligence and Statistics. Chia Laguna Resort, Sardinia, Italy, 249--256.
[10]
A. Graves, N. Jaitly, and A. Mohamed. 2013. Hybrid speech recognition with Deep Bidirectional LSTM. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic, December 8-12, 2013. 273--278.
[11]
A. Graves, A. Mohamed, and G. Hinton. 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC, Canada, 6645--6649.
[12]
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. 2016. Hybrid computing using a neural network with dynamic external memory. Nature, Vol. 538 (12 Oct 2016), 471.
[13]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput., Vol. 9, 8 (Nov. 1997), 1735--1780.
[14]
Yacine Jernite, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Variable computation in recurrent neural networks. 5th International Conference on Learning Representations, (ICLR), Toulon, France, April 24-26, 2017, Conference Track Proceedings.
[15]
Mohammad Khajah, Robert V. Lindsey, and Michael C. Mozer. 2016. How Deep is Knowledge Tracing? In Proceedings of the 9th International Conference on Educational Data Mining, (EDM), Raleigh, North Carolina, USA, June 29 - July 2, 2016.
[16]
Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A Method for Stochastic Optimization. In international conference on learning representations (ICLR).
[17]
George J. Klir and Bo Yuan. 1995. Fuzzy Sets and Fuzzy Logic: Theory and Applications. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
[18]
Aditya Kusupati, Manish Singh, Kush Bhatia, Ashish Kumar, Prateek Jain, and Manik Varma. 2018. FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems, (NeurIPS), 3-8 December 2018, Montréal, Canada. 9017--9028.
[19]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, Vol. 521, 7553 (2015), 436.
[20]
Charles X. Ling, Jin Huang, and Harry Zhang. 2003. AUC: A Statistically Consistent and More Discriminating Measure Than Accuracy. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI). San Francisco, CA, USA, 519--524.
[21]
Jun Liu, Amir Shahroudy, Dong Xu, and Gang Wang. 2016. Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition. In 14th European Conference on Computer Vision, (ECCV), Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III. Cham, 816--833.
[22]
Danilo P. Mandic and Jonathon Chambers. 2001. Recurrent Neural Networks for Prediction: Learning Algorithms,Architectures and Stability. John Wiley & Sons, Inc., New York, NY, USA.
[23]
Zachary A. Pardos and Neil T. Heffernan. 2010. Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing. In Proceedings of the 18th International Conference on User Modeling, Adaptation, and Personalization (UMAP). Berlin, Heidelberg, 255--266.
[24]
Zachary A. Pardos and Neil T. Heffernan. 2011. KT-IDEM: Introducing Item Difficulty to the Knowledge Tracing Model. In Proceedings of the 19th International Conference on User Modeling, Adaption, and Personalization (UMAP). Berlin, Heidelberg, 243--254.
[25]
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the Difficulty of Training Recurrent Neural Networks. In Proceedings of the 30th International Conference on International Conference on Machine Learning (ICML). Atlanta, Georgia, USA, III-1310--III-1318.
[26]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, and Jascha Sohl-Dickstein. 2015. Deep Knowledge Tracing. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada (NeurIPS). Cambridge, MA, USA, 505--513.
[27]
Christopher James Piech. 2016. Uncovering Patterns in Student Work: Machine Learning to Understand Human Learning. Ph.D. Dissertation. Stanford University.
[28]
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-learning with Memory-augmented Neural Networks. In Proceedings of the 33nd International Conference on Machine Learning, (ICML), New York City, NY, USA, June 19-24, 2016. 1842--1850.
[29]
Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural Netw., Vol. 61, C (Jan. 2015), 85--117.
[30]
M. Schuster and K. K. Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, Vol. 45, 11 (Nov 1997), 2673--2681.
[31]
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems, December 8-13 2014, Montreal, Quebec, Canada (NeurIPS). Cambridge, MA, USA, 3104--3112.
[32]
Michael Villano. 1992. Probabilistic Student Models: Bayesian Belief Networks and Knowledge Space Theory. In Proceedings of the Second International Conference on Intelligent Tutoring Systems (ITS). London, UK, UK, 491--498.
[33]
P. Wang, A. Jiang, X. Liu, J. Shang, and L. Zhang. 2018. LSTM-Based EEG Classification in Motor Imagery Tasks. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 26, 11 (Nov 2018), 2086--2095.
[34]
Adams Wei Yu, Hongrae Lee, and Quoc V. Le. 2017. Learning to Skim Text. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, (ACL), Vancouver, Canada, July 30 - August 4, (Volume 1: Long Papers). 1880--1890.
[35]
Michael V. Yudelson, Kenneth R. Koedinger, and Geoffrey J. Gordon. 2013. Individualized bayesian knowledge tracing models. Artificial Intelligence in Education - 16th International Conference, (AIED), Memphis, TN, USA, July 9-13, 2013. Proceedings. 171--180.
[36]
Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic Key-Value Memory Networks for Knowledge Tracing. In Proceedings of the 26th International Conference on World Wide Web (WWW). Republic and Canton of Geneva, Switzerland, 765--774.

Cited By

View all
  • (2025)AAKT: Enhancing Knowledge Tracing With Alternate Autoregressive ModelingIEEE Transactions on Learning Technologies10.1109/TLT.2024.352189818(25-38)Online publication date: 1-Jan-2025
  • (2025)Enhanced Knowledge Tracing With Learnable FilterIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.345213012:1(198-209)Online publication date: Feb-2025
  • (2025)LGS-KT: Integrating logical and grammatical skills for effective programming knowledge tracingNeural Networks10.1016/j.neunet.2025.107164185(107164)Online publication date: May-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval
July 2019
1512 pages
ISBN:9781450361729
DOI:10.1145/3331184
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. deep learning
  2. key-value memory
  3. knowledge tracing
  4. memory networks
  5. sequence modelling

Qualifiers

  • Research-article

Funding Sources

  • ANU Vice-Chancellor's Teaching Enhancement
  • Australian government higher education

Conference

SIGIR '19
Sponsor:

Acceptance Rates

SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;
Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)141
  • Downloads (Last 6 weeks)20
Reflects downloads up to 08 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)AAKT: Enhancing Knowledge Tracing With Alternate Autoregressive ModelingIEEE Transactions on Learning Technologies10.1109/TLT.2024.352189818(25-38)Online publication date: 1-Jan-2025
  • (2025)Enhanced Knowledge Tracing With Learnable FilterIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.345213012:1(198-209)Online publication date: Feb-2025
  • (2025)LGS-KT: Integrating logical and grammatical skills for effective programming knowledge tracingNeural Networks10.1016/j.neunet.2025.107164185(107164)Online publication date: May-2025
  • (2025)Dual-view multi-scale cognitive representation for deep knowledge tracingKnowledge-Based Systems10.1016/j.knosys.2025.113010310(113010)Online publication date: Feb-2025
  • (2025)MAHKT: Knowledge tracing with multi-association heterogeneous graph embedding based on knowledge transferKnowledge-Based Systems10.1016/j.knosys.2025.112958310(112958)Online publication date: Feb-2025
  • (2025)Multi-Granularity Ensemble Interaction Graph Modeling for Knowledge TracingKnowledge-Based Systems10.1016/j.knosys.2024.112834309(112834)Online publication date: Jan-2025
  • (2025)Enhancing learning process modeling for session-aware knowledge tracingKnowledge-Based Systems10.1016/j.knosys.2024.112740309(112740)Online publication date: Jan-2025
  • (2025)csKT: Addressing cold-start problem in knowledge tracing via kernel bias and cone attentionExpert Systems with Applications10.1016/j.eswa.2024.125988266(125988)Online publication date: Mar-2025
  • (2025)EduStudio: towards a unified library for student cognitive modelingFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-024-40372-319:8Online publication date: 1-Aug-2025
  • (2024)An Enhanced Deep Knowledge Tracing Model via Multiband Attention and Quantized Question EmbeddingApplied Sciences10.3390/app1408342514:8(3425)Online publication date: 18-Apr-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media