research-article

Enhancing Knowledge Tracing via Adversarial Training

Authors:

Jun SunAuthors Info & Claims

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Pages 367 - 375

https://doi.org/10.1145/3474085.3475554

Published: 17 October 2021 Publication History

Abstract

We study the problem of knowledge tracing (KT) where the goal is to trace the students' knowledge mastery over time so as to make predictions on their future performance. Owing to the good representation capacity of deep neural networks (DNNs), recent advances on KT have increasingly concentrated on exploring DNNs to improve the performance of KT. However, we empirically reveal that the DNNs based KT models may run the risk of overfitting, especially on small datasets, leading to limited generalization. In this paper, by leveraging the current advances in adversarial training (AT), we propose an efficient AT based KT method (ATKT) to enhance KT model's generalization and thus push the limit of KT. Specifically, we first construct adversarial perturbations and add them on the original interaction embeddings as adversarial examples. The original and adversarial examples are further used to jointly train the KT model, forcing it is not only to be robust to the adversarial examples, but also to enhance the generalization over the original ones. To better implement AT, we then present an efficient attentive-LSTM model as KT backbone, where the key is a proposed knowledge hidden state attention module that adaptively aggregates information from previous knowledge hidden states while simultaneously highlighting the importance of current knowledge hidden state to make a more accurate prediction. Extensive experiments on four public benchmark datasets demonstrate that our ATKT achieves new state-of-the-art performance. Code is available at: https://github.com/xiaopengguo/ATKT.

References

[1]

Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web. 687--698.

Digital Library

[2]

Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim vS rndi?, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. 2013. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases. Springer, 387--402.

Digital Library

[3]

Hao Cen, Kenneth Koedinger, and Brian Junker. 2006. Learning factors analysis--a general method for cognitive model evaluation and improvement. In International Conference on Intelligent Tutoring Systems. Springer, 164--175.

Digital Library

[4]

Mauro Conti, Roberto Di Pietro, Luigi V. Mancini, and Alessandro Mei. 2009. (old) Distributed data source verification in wireless sensor networks. Inf. Fusion, Vol. 10, 4 (2009), 342--353. https://doi.org/10.1016/j.inffus.2009.01.002

Digital Library

[5]

Albert T Corbett and John R Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, Vol. 4, 4 (1994), 253--278.

[6]

Ryan SJ d Baker, Albert T Corbett, and Vincent Aleven. 2008. More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In International conference on intelligent tutoring systems. Springer, 406--415.

Digital Library

[7]

Xin Dong, Yaxin Zhu, Yupeng Zhang, Zuohui Fu, Dongkuan Xu, Sen Yang, and Gerard De Melo. 2020. Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1541--1544.

Digital Library

[8]

Aritra Ghosh, Neil Heffernan, and Andrew S Lan. 2020. Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2330--2339.

Digital Library

[9]

Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations. http://arxiv.org/abs/1412.6572

[10]

William J Hawkins, Neil T Heffernan, and Ryan SJD Baker. 2014. Learning Bayesian knowledge tracing parameters with a knowledge heuristic and empirical probabilities. In International Conference on Intelligent Tutoring Systems. Springer, 150--155.

Digital Library

[11]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

Digital Library

[12]

Tanja Käser, Severin Klingler, Alexander G Schwing, and Markus Gross. 2017. Dynamic Bayesian networks for student modeling. IEEE Transactions on Learning Technologies, Vol. 10, 4 (2017), 450--462.

Digital Library

[13]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6980

[14]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012), 1097--1105.

Digital Library

[15]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, Vol. 521, 7553 (2015), 436--444.

[16]

Jinseok Lee and Dit-Yan Yeung. 2019. Knowledge query network for knowledge tracing: How knowledge interacts with skills. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge. 491--500.

Digital Library

[17]

Kai Liu, Xin Liu, An Yang, Jing Liu, Jinsong Su, Sujian Li, and Qiaoqiao She. 2020. A robust adversarial training approach to machine reading comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 8392--8400.

[18]

Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Hui Xiong, Yu Su, and Guoping Hu. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, Vol. 33, 1 (2019), 100--115.

Digital Library

[19]

Sein Minn, Yi Yu, Michel C Desmarais, Feida Zhu, and Jill-Jenn Vie. 2018. Deep knowledge tracing and dynamic student classification for knowledge tracing. In 2018 IEEE International conference on data mining (ICDM). IEEE, 1182--1187.

[20]

Takeru Miyato, Andrew M. Dai, and Ian Goodfellow. 2017. Adversarial Training Methods for Semi-Supervised Text Classification. ICLR (2017). https://arxiv.org/abs/1605.07725

[21]

Koki Nagatani, Qian Zhang, Masahiro Sato, Yan-Ying Chen, Francine Chen, and Tomoko Ohkuma. 2019. Augmenting knowledge tracing by considering forgetting behavior. In The world wide web conference. 3101--3107.

Digital Library

[22]

Shalini Pandey and George Karypis. 2019. A self-attentive model for knowledge tracing (EDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining). International Educational Data Mining Society, 384--389.

[23]

Shalini Pandey and Jaideep Srivastava. 2020. RKT: Relation-Aware Self-Attention for Knowledge Tracing. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1205--1214.

Digital Library

[24]

Zachary A Pardos and Neil T Heffernan. 2011. KT-IDEM: Introducing item difficulty to the knowledge tracing model. In International conference on user modeling, adaptation, and personalization. Springer, 243--254.

Digital Library

[25]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc.

Digital Library

[26]

Philip I Pavlik Jr, Hao Cen, and Kenneth R Koedinger. 2009. Performance Factors Analysis--A New Alternative to Knowledge Tracing. Online Submission (2009).

Digital Library

[27]

Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep Knowledge Tracing. In Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2015/file/bac9162b47c56fc8a4d2a519803d51b3-Paper.pdf

Digital Library

[28]

Lutz Prechelt. 1998. Automatic early stopping using cross validation: quantifying the criteria. Neural Networks, Vol. 11, 4 (1998), 761--767.

Digital Library

[29]

David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors. nature, Vol. 323, 6088 (1986), 533--536.

[30]

Motoki Sato, Jun Suzuki, and Shun Kiyono. 2019. Effective adversarial regularization for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 204--210.

[31]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations. http://arxiv.org/abs/1312.6199

[32]

Jinhui Tang, Xiaoyu Du, Xiangnan He, Fajie Yuan, Qi Tian, and Tat-Seng Chua. 2019. Adversarial training towards robust multimedia recommender system. IEEE Transactions on Knowledge and Data Engineering, Vol. 32, 5 (2019), 855--867.

[33]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017).

[34]

Tianqi Wang, Fenglong Ma, and Jing Gao. 2019. Deep hierarchical knowledge tracing. In Proceedings of the 12th International Conference on Educational Data Mining.

[35]

Xin Wang, Wei Huang, Qi Liu, Yu Yin, Zhenya Huang, Le Wu, Jianhui Ma, and Xue Wang. 2020. Fine-Grained Similarity Measurement between Educational Videos and Exercises. In Proceedings of the 28th ACM International Conference on Multimedia. 331--339.

Digital Library

[36]

Yi Wu, David Bamman, and Stuart Russell. 2017. Adversarial training for relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 1778--1783.

[37]

Zhengyang Wu, Ming Li, Yong Tang, and Qingyu Liang. 2020. Exercise recommendation based on knowledge concept prediction. Knowledge-Based Systems, Vol. 210 (2020), 106481.

[38]

Ziqing Yang, Yiming Cui, Wanxiang Che, Ting Liu, Shijin Wang, and Guoping Hu. 2019. Improving machine reading comprehension via adversarial training. arXiv preprint arXiv:1911.03614 (2019).

[39]

Michihiro Yasunaga, Jungo Kasai, and Dragomir Radev. 2018. Robust Multilingual Part-of-Speech Tagging via Adversarial Training. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, New Orleans, Louisiana, 976--986.https://doi.org/10.18653/v1/N18-1089

[40]

Chun-Kit Yeung and Dit-Yan Yeung. 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. 1--10.

Digital Library

[41]

Michael V Yudelson, Kenneth R Koedinger, and Geoffrey J Gordon. 2013. Individualized bayesian knowledge tracing models. In International conference on artificial intelligence in education. Springer, 171--180.

[42]

Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017a. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on World Wide Web. 765--774.

Digital Library

[43]

Liang Zhang, Xiaolu Xiong, Siyuan Zhao, Anthony Botelho, and Neil T Heffernan. 2017b. Incorporating rich features into deep knowledge tracing. In Proceedings of the fourth (2017) ACM conference on learning@ scale. 169--172.

Digital Library

Cited By

Zhang HLiu ZShang CLi DJiang Y(2024)A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing ModelsACM Transactions on Knowledge Discovery from Data10.1145/3674840Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674840
Ma HYang YQin CYu XYang SZhang XZhu HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)HD-KT: Advancing Robust Knowledge Tracing via Anomalous Learning Interaction DetectionProceedings of the ACM on Web Conference 202410.1145/3589334.3645718(4479-4488)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645718
Sun JYu FWan QLi QLiu SShen XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Interpretable Knowledge Tracing with Multiscale State RepresentationProceedings of the ACM on Web Conference 202410.1145/3589334.3645373(3265-3276)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645373
Show More Cited By

Index Terms

Enhancing Knowledge Tracing via Adversarial Training
1. Applied computing
  1. Education
    1. Computer-assisted instruction
    2. Learning management systems

Recommendations

A hybrid adversarial training for deep learning model and denoising network resistant to adversarial examples
Abstract
Deep neural networks (DNNs) are vulnerable to adversarial attacks that generate adversarial examples by adding small perturbations to the clean images. To combat adversarial attacks, the two main defense methods used are denoising and adversarial ...
Wavelet regularization benefits adversarial training
Abstract
Adversarial training methods are frequently-used empirical defense methods against adversarial examples. While many regularization techniques demonstrate effectiveness when combined with adversarial training, these methods typically ...
LADDER: Latent boundary-guided adversarial training
Abstract
Deep Neural Networks (DNNs) have recently achieved great success in many classification tasks. Unfortunately, they are vulnerable to adversarial attacks that generate adversarial examples with a small perturbation to fool DNN models, especially in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

October 2021

5796 pages

ISBN:9781450386517

DOI:10.1145/3474085

General Chairs:
Heng Tao Shen
University of Electronic Science&Technology of China, China
,
Yueting Zhuang
Zhejiang University, China
,
John R. Smith
IBM, USA
,
Program Chairs:
Yang Yang
University of Electronic Science and Technology of China, China
,
Pablo Cesar
CWI&TU Delft, The Netherlands
,
Florian Metze
FACEBOOK, Inc., USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20 - 24, 2021

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
526
Total Downloads

Downloads (Last 12 months)125
Downloads (Last 6 weeks)6

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HLiu ZShang CLi DJiang Y(2024)A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing ModelsACM Transactions on Knowledge Discovery from Data10.1145/3674840Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674840
Ma HYang YQin CYu XYang SZhang XZhu HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)HD-KT: Advancing Robust Knowledge Tracing via Anomalous Learning Interaction DetectionProceedings of the ACM on Web Conference 202410.1145/3589334.3645718(4479-4488)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645718
Sun JYu FWan QLi QLiu SShen XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Interpretable Knowledge Tracing with Multiscale State RepresentationProceedings of the ACM on Web Conference 202410.1145/3589334.3645373(3265-3276)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645373
Zu SCai STang WWang CLi LShen J(2024)GuessKT: Improving Knowledge Tracing via Considering Guess BehaviorsICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447277(12811-12815)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447277
Hegde AS AC H AB V VHegde C(2024)Enhanced Transformer: Knowledge Tracing with Incorporation of Temporal Features2024 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI)10.1109/IATMSI60426.2024.10502970(1-6)Online publication date: 14-Mar-2024
https://doi.org/10.1109/IATMSI60426.2024.10502970
Yan YGuan ZWang XWei YYang Z(2024)Knowledge Tracing with Soft Labels Via Knowledge Distillation and IRT-Based Modeling2024 IEEE 3rd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA)10.1109/EEBDA60612.2024.10486040(382-386)Online publication date: 27-Feb-2024
https://doi.org/10.1109/EEBDA60612.2024.10486040
Ke FWang WTan WDu LJin YHuang YYin H(2024)HiTSKTKnowledge-Based Systems10.1016/j.knosys.2023.111300284:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2023.111300
Liu DGuo Lzhang XLi Y(2024)ETVKT: Enhanced Training Vector for Knowledge TracingAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5612-4_41(474-481)Online publication date: 31-Jul-2024
https://doi.org/10.1007/978-981-97-5612-4_41
Guo XShu MHuang ZLiu JSun J(2024)Programming Knowledge Tracing with Context and Structure IntegrationKnowledge Science, Engineering and Management10.1007/978-981-97-5492-2_10(124-135)Online publication date: 26-Jul-2024
https://doi.org/10.1007/978-981-97-5492-2_10
Zhan BGuo TLi XHou MLiang QGao BLuo WLiu Z(2024)Knowledge Tracing as Language Processing: A Large-Scale Autoregressive ParadigmArtificial Intelligence in Education10.1007/978-3-031-64302-6_13(177-191)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64302-6_13
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents