Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1609/aaai.v37i12.26661guideproceedingsArticle/Chapter ViewAbstractPublication PagesConference Proceedingsacm-pubtype
research-article

Improving interpretability of deep sequential knowledge tracing models with question-centric cognitive representations

Published: 07 February 2023 Publication History
  • Get Citation Alerts
  • Abstract

    Knowledge tracing (KT) is a crucial technique to predict students' future performance by observing their historical learning processes. Due to the powerful representation ability of deep neural networks, remarkable progress has been made by using deep learning techniques to solve the KT problem. The majority of existing approaches rely on the homogeneous question assumption that questions have equivalent contributions if they share the same set of knowledge components. Unfortunately, this assumption is inaccurate in real-world educational scenarios. Furthermore, it is very challenging to interpret the prediction results from the existing deep learning based KT models. Therefore, in this paper, we present QIKT, a question-centric interpretable KT model to address the above challenges. The proposed QIKT approach explicitly models students' knowledge state variations at a fine-grained level with question-sensitive cognitive representations that are jointly learned from a question-centric knowledge acquisition module and a question-centric problem solving module. Meanwhile, the QIKT utilizes an item response theory based prediction layer to generate interpretable prediction results. The proposed QIKT model is evaluated on three public real-world educational datasets. The results demonstrate that our approach is superior on the KT prediction task, and it outperforms a wide range of deep learning based KT models in terms of prediction accuracy with better model interpretability. To encourage reproducible results, we have provided all the datasets and code at https://pykt.org/.

    References

    [1]
    Abdelrahman, G.; and Wang, Q. 2019. Knowledge tracing with sequential key-value memory networks. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 175-184.
    [2]
    Cen, H.; Koedinger, K.; and Junker, B. 2006. Learning factors analysis-a general method for cognitive model evaluation and improvement. In International Conference on Intelligent Tutoring Systems, 164-175. Springer.
    [3]
    Chen, P.; Lu, Y.; Zheng, V. W.; and Pian, Y. 2018. Prerequisite-driven deep knowledge tracing. In 2018 IEEE International Conference on Data Mining, 39-48. IEEE.
    [4]
    Choi, Y.; Lee, Y.; Cho, J.; Baek, J.; Kim, B.; Cha, Y.; Shin, D.; Bae, C.; and Heo, J. 2020. Towards an appropriate query, key, and value computation for knowledge tracing. In Proceedings of the Seventh ACM Conference on Learning@Scale, 341-344.
    [5]
    Converse, G.; Pu, S.; and Oliveira, S. 2021. Incorporating item response theory into knowledge tracing. In International Conference on Artificial Intelligence in Education, 114-118. Springer.
    [6]
    Corbett, A. T.; and Anderson, J. R. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction, 4(4): 253-278.
    [7]
    Ghosh, A.; Heffernan, N.; and Lan, A. S. 2020. Context-Aware Attentive Knowledge Tracing. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
    [8]
    Guo, X.; Huang, Z.; Gao, J.; Shang, M.; Shu, M.; and Sun, J. 2021. Enhancing Knowledge Tracing via Adversarial Training. In Proceedings of the 29th ACM International Conference on Multimedia, 367-375.
    [9]
    Kaser, T.; Klingler, S.; Schwing, A. G.; and Gross, M. 2017. Dynamic Bayesian networks for student modeling. IEEE Transactions on Learning Technologies, 10(4): 450-462.
    [10]
    Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations.
    [11]
    Lavoue, E.; Monterrat, B.; Desmarais, M.; and George, S. 2018. Adaptive gamification for learning environments. IEEE Transactions on Learning Technologies, 12(1): 16-28.
    [12]
    Lee, J.; and Yeung, D.-Y. 2019. Knowledge query network for knowledge tracing: How knowledge interacts with skills. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 491-500.
    [13]
    Liu, Q.; Huang, Z.; Yin, Y.; Chen, E.; Xiong, H.; Su, Y.; and Hu, G. 2019. EKT: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, 33(1): 100-115.
    [14]
    Liu, T.; Fang, Q.; Ding, W.; Li, H.; Wu, Z.; and Liu, Z. 2021a. Mathematical Word Problem Generation from Commonsense Knowledge Graph and Equations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 4225-4240.
    [15]
    Liu, Y.; Yang, Y.; Chen, X.; Shen, J.; Zhang, H.; and Yu, Y. 2021b. Improving knowledge tracing via pre-training question embeddings. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, 1556-1562.
    [16]
    Liu, Z.; Liu, Q.; Chen, J.; Huang, S.; Gao, B.; Luo, W.; and Weng, J. 2023a. Enhancing Deep Knowledge Tracing with Auxiliary Tasks. In Proceedings of the 2023 World Wide Web Conference.
    [17]
    Liu, Z.; Liu, Q.; Chen, J.; Huang, S.; and Luo, W. 2023b. simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing. In International Conference on Learning Representations.
    [18]
    Liu, Z.; Liu, Q.; Chen, J.; Huang, S.; Tang, J.; and Luo, W. 2022. PYKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models. In Thirty-sixth Conference on Neural Information Processing Systems.
    [19]
    Long, T.; Liu, Y.; Shen, J.; Zhang, W.; and Yu, Y. 2021. Tracing Knowledge State with Individual Cognition and Acquisition Estimation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 173-182.
    [20]
    Lu, Y.; Wang, D.; Chen, P.; Meng, Q.; and Yu, S. 2022. Interpreting Deep Learning Models for Knowledge Tracing. International Journal of Artificial Intelligence in Education, 1-24.
    [21]
    Lu, Y.; Wang, D.; Meng, Q.; and Chen, P. 2020. Towards interpretable deep learning models for knowledge tracing. In International Conference on Artificial Intelligence in Education, 185-190. Springer.
    [22]
    Minn, S.; Yu, Y.; Desmarais, M. C.; Zhu, F.; and Vie, J.-J. 2018. Deep knowledge tracing and dynamic student classification for knowledge tracing. In 2018 IEEE International Conference on Data Mining, 1182-1187. IEEE.
    [23]
    Nagatani, K.; Zhang, Q.; Sato, M.; Chen, Y.-Y.; Chen, F.; and Ohkuma, T. 2019. Augmenting knowledge tracing by considering forgetting behavior. In The World Wide Web Conference, 3101-3107.
    [24]
    Nakagawa, H.; Iwasawa, Y.; and Matsuo, Y. 2019. Graph-based knowledge tracing: modeling student proficiency using graph neural network. In 2019 IEEE/WIC/ACM International Conference on Web Intelligence, 156-163. IEEE.
    [25]
    Pandey, S.; and Karypis, G. 2019. A self-attentive model for knowledge tracing. In 12th International Conference on Educational Data Mining, 384-389. International Educational Data Mining Society.
    [26]
    Pandey, S.; and Srivastava, J. 2020. RKT: relation-aware self-attention for knowledge tracing. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 1205-1214.
    [27]
    Piech, C.; Bassen, J.; Huang, J.; Ganguli, S.; Sahami, M.; Guibas, L. J.; and Sohl-Dickstein, J. 2015. Deep knowledge tracing. Advances in neural information processing systems, 28.
    [28]
    Pu, S.; Yudelson, M.; Ou, L.; and Huang, Y. 2020. Deep knowledge tracing with transformers. In International Conference on Artificial Intelligence in Education, 252-256. Springer.
    [29]
    Pu, Y.; Wu, W.; Peng, T.; Liu, F.; Liang, Y.; Yu, X.; Chen, R.; and Feng, P. 2022. EAKT: Embedding Cognitive Framework with Attention for Interpretable Knowledge Tracing. Scientific Reports.
    [30]
    Shen, S.; Liu, Q.; Chen, E.; Huang, Z.; Huang, W.; Yin, Y.; Su, Y.; and Wang, S. 2021. Learning Process-consistent Knowledge Tracing. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 1452-1460.
    [31]
    Shen, S.; Liu, Q.; Chen, E.; Wu, H.; Huang, Z.; Zhao, W.; Su, Y.; Ma, H.; and Wang, S. 2020. Convolutional knowledge tracing: Modeling individualization in student learning process. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 1857-1860.
    [32]
    Sonkar, S.; Waters, A. E.; Lan, A. S.; Grimaldi, P. J.; and Baraniuk, R. G. 2020. qDKT: Question-centric deep knowledge tracing. In Proceedings of The 13th International Conference on Educational Data Mining, 677-681.
    [33]
    Su, Y.; Liu, Q.; Liu, Q.; Huang, Z.; Yin, Y.; Chen, E.; Ding, C.; Wei, S.; and Hu, G. 2018. Exercise-enhanced sequential modeling for student performance prediction. In Proceedings of the AAAI Conference on Artificial Intelligence.
    [34]
    Tong, S.; Liu, Q.; Huang, W.; Huang, Z.; Chen, E.; Liu, C.; Ma, H.; and Wang, S. 2020. Structure-based Knowledge Tracing: An Influence Propagation View. In 2020 IEEE International Conference on Data Mining, 541-550. IEEE.
    [35]
    Wang, F.; Liu, Q.; Chen, E.; Huang, Z.; Chen, Y.; Yin, Y.; Huang, Z.; and Wang, S. 2020a. Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, 6153-6161.
    [36]
    Wang, Z.; Feng, X.; Tang, J.; Huang, G. Y.; and Liu, Z. 2019. Deep knowledge tracing with side information. In International Conference on Artificial Intelligence in Education, 303-308. Springer.
    [37]
    Wang, Z.; Lamb, A.; Saveliev, E.; Cameron, P.; Zaykov, Y.; Hernández-Lobato, J. M.; Turner, R. E.; Baraniuk, R. G.; Barton, C.; Jones, S. P.; et al. 2020b. Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge. ArXiv preprint, abs/2007.12061.
    [38]
    Yang, Y.; Shen, J.; Qu, Y.; Liu, Y.; Wang, K.; Zhu, Y.; Zhang, W.; and Yu, Y. 2020. GIKT: a graph-based interaction model for knowledge tracing. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 299-315. Springer.
    [39]
    Yeung, C.-K. 2019. Deep-IRT: Make deep learning based knowledge tracing explainable using item response theory. arXiv preprint arXiv:1904.11738.
    [40]
    Yeung, C.-K.; and Yeung, D.-Y. 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 1-10.
    [41]
    Zhang, J.; Shi, X.; King, I.; and Yeung, D. Y. 2017. Dynamic Key-Value Memory Networks for Knowledge Tracing. In Proceedings of the 26th International Conference on World Wide Web, 765.
    [42]
    Zhang, M.; Zhu, X.; Zhang, C.; Ji, Y.; Pan, F.; and Yin, C. 2021. Multi-Factors Aware Dual-Attentional Knowledge Tracing. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2588-2597.
    [43]
    Zhao, J.; Bhatt, S.; Thille, C.; Zimmaro, D.; and Gattani, N. 2020. Interpretable personalized knowledge tracing and next learning activity recommendation. In Proceedings of the Seventh ACM Conference on Learning@Scale, 325-328.

    Cited By

    View all
    • (2024)Question Difficulty Consistent Knowledge TracingProceedings of the ACM on Web Conference 202410.1145/3589334.3645582(4239-4248)Online publication date: 13-May-2024
    • (2024)Interpretable Knowledge Tracing with Multiscale State RepresentationProceedings of the ACM on Web Conference 202410.1145/3589334.3645373(3265-3276)Online publication date: 13-May-2024
    • (2023)Enhancing Deep Knowledge Tracing with Auxiliary TasksProceedings of the ACM Web Conference 202310.1145/3543507.3583866(4178-4187)Online publication date: 30-Apr-2023
    • Show More Cited By

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Guide Proceedings
    AAAI'23/IAAI'23/EAAI'23: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence
    February 2023
    16496 pages
    ISBN:978-1-57735-880-0

    Sponsors

    • Association for the Advancement of Artificial Intelligence

    Publisher

    AAAI Press

    Publication History

    Published: 07 February 2023

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Question Difficulty Consistent Knowledge TracingProceedings of the ACM on Web Conference 202410.1145/3589334.3645582(4239-4248)Online publication date: 13-May-2024
    • (2024)Interpretable Knowledge Tracing with Multiscale State RepresentationProceedings of the ACM on Web Conference 202410.1145/3589334.3645373(3265-3276)Online publication date: 13-May-2024
    • (2023)Enhancing Deep Knowledge Tracing with Auxiliary TasksProceedings of the ACM Web Conference 202310.1145/3543507.3583866(4178-4187)Online publication date: 30-Apr-2023
    • (2023)Towards Robust Knowledge Tracing Models via k-Sparse AttentionProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592073(2441-2445)Online publication date: 19-Jul-2023
    • (2023)Recent Advances on Deep Learning based Knowledge TracingProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3575790(1295-1296)Online publication date: 27-Feb-2023

    View Options

    View options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media