research-article

RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems

Authors:

Shijin Wang, and

Yu SuAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

Pages 501 - 510

https://doi.org/10.1145/3404835.3462932

Published: 11 July 2021 Publication History

Abstract

Cognitive diagnosis (CD) is a fundamental issue in intelligent educational settings, which aims to discover the mastery levels of students on different knowledge concepts. In general, most previous works consider it as an inter-layer interaction modeling problem, e.g., student-exercise interactions in IRT or student-concept interactions in DINA, while the inner-layer structural relations, such as educational interdependencies among concepts, are still underexplored. Furthermore, there is a lack of comprehensive modeling for the student-exercise-concept hierarchical relations in CD systems. To this end, in this paper, we present a novel Relation map driven Cognitive Diagnosis (RCD) framework, uniformly modeling the interactive and structural relations via a multi-layer student-exercise-concept relation map. Specifically, we first represent students, exercises and concepts as individual nodes in a hierarchical layout, and construct three well-defined local relation maps to incorporate inter- and inner-layer relations, including a student-exercise interaction map, a concept-exercise correlation map and a concept dependency map. Then, we leverage a multi-level attention network to integrate node-level relation aggregation inside each local map and balance map-level aggregation across different maps. Finally, we design an extendable diagnosis function to predict students' performance and jointly train the networks. Extensive experimental results on real-world datasets clearly show the effectiveness and extendibility of our RCD in both diagnosis accuracy improvement and relation-aware representation learning.

Supplementary Material

MP4 File (RCD.mp4)

Presentation video of SIGIR-2021 full paper "RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems"

Download
64.75 MB

References

[1]

Terry A Ackerman, Mark J Gierl, and Cindy M Walker. 2003. Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practice, Vol. 22, 3 (2003), 37--51.

[2]

Raymond J Adams, Mark Wilson, and Wen-chung Wang. 1997. The multidimensional random coefficients multinomial logit model. Applied psychological measurement, Vol. 21, 1 (1997), 1--23.

[3]

Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web. 687--698.

Digital Library

[4]

Haoyang Bi, Haiping Ma, Zhenya Huang, Yu Yin, Qi Liu, Enhong Chen, Yu Su, and Shijin Wang. 2020. Quality meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 42--51.

[5]

Andrew P Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition, Vol. 30, 7 (1997), 1145--1159.

[6]

Hugh Burns, Carol A Luckhardt, James W Parlett, and Carol L Redfield. 2014. Intelligent tutoring systems: Evolutions in design .Psychology Press.

[7]

Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 27--34.

[8]

Penghe Chen, Yu Lu, Vincent W Zheng, and Yang Pian. 2018. Prerequisite-driven deep knowledge tracing. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 39--48.

[9]

Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2018. A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering, Vol. 31, 5 (2018), 833--852.

[10]

Jimmy De La Torre. 2009. DINA model and parameter estimation: A didactic. Journal of educational and behavioral statistics, Vol. 34, 1 (2009), 115--130.

[11]

Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 135--144.

Digital Library

[12]

Henry C Ellis. 1965. The transfer of learning. (1965).

[13]

Susan E Embretson and Steven P Reise. 2013. Item response theory .Psychology Press.

[14]

Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1416--1424.

Digital Library

[15]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855--864.

Digital Library

[16]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in neural information processing systems. 1024--1034.

[17]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 639--648.

Digital Library

[18]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[19]

Xiaoqing Huang, Qi Liu, Chao Wang, Haoyu Han, Jianhui Ma, Enhong Chen, Yu Su, and Shijin Wang. 2019 a. Constructing Educational Concept Maps with Multiple Relationships from Multi-Source Data. In 2019 IEEE ICDM. IEEE, 1108--1113.

[20]

Zhenya Huang, Qi Liu, Yuying Chen, Le Wu, Keli Xiao, Enhong Chen, Haiping Ma, and Guoping Hu. 2020. Learning or forgetting? A dynamic approach for tracking the knowledge proficiency of students. ACM Transactions on Information Systems (TOIS), Vol. 38, 2 (2020), 1--33.

Digital Library

[21]

Zhenya Huang, Qi Liu, Chengxiang Zhai, Yu Yin, Enhong Chen, Weibo Gao, and Guoping Hu. 2019 b. Exploring multi-objective exercise recommendations in online education systems. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1261--1270.

Digital Library

[22]

Mohammad Khajah, Rowan Wing, Robert Lindsey, and Michael Mozer. 2014. Integrating latent-factor and knowledge-tracing models to predict individual differences in learning. In Educational Data Mining 2014. Citeseer.

[23]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[24]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009), 30--37.

Digital Library

[25]

George D Kuh, Jillian Kinzie, Jennifer A Buckley, Brian K Bridges, and John C Hayek. 2011. Piecing together the student success puzzle: Research, propositions, and recommendations: ASHE higher education report. Vol. 116. John Wiley & Sons.

[26]

Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Hui Xiong, Yu Su, and Guoping Hu. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, Vol. 33, 1 (2019), 100--115.

Digital Library

[27]

Qi Liu, Runze Wu, Enhong Chen, Guandong Xu, Yu Su, Zhigang Chen, and Guoping Hu. 2018. Fuzzy cognitive diagnosis for modelling examinee performance. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 9, 4 (2018), 1--26.

Digital Library

[28]

Andriy Mnih and Russ R Salakhutdinov. 2007. Probabilistic matrix factorization. Advances in neural information processing systems, Vol. 20 (2007), 1257--1264.

[29]

Hiromi Nakagawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Graph-based Knowledge Tracing: Modeling Student Proficiency Using Graph Neural Network. In 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI). IEEE, 156--163.

[30]

Joseph D Novak. 2010. Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations .Routledge.

[31]

Shalini Pandey and Jaideep Srivastava. 2020. RKT: Relation-Aware Self-Attention for Knowledge Tracing. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1205--1214.

Digital Library

[32]

Hongbin Pei, Bo Yang, Jiming Liu, and Lei Dong. 2017. Group sparse bayesian learning for active surveillance on epidemic dynamics. arXiv preprint arXiv:1712.00328 (2017).

[33]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701--710.

Digital Library

[34]

Jean Piaget. 1985. The equilibration of cognitive structures: The central problem of intellectual development .University of Chicago Press.

[35]

William F Pinar, William M Reynolds, Patrick Slattery, Peter M Taubman, et al. 1995. Understanding curriculum: An introduction to the study of historical and contemporary curriculum discourses. Vol. 17. Peter Lang.

[36]

Mark D Reckase. 2009. Multidimensional item response theory models. In Multidimensional item response theory. Springer, 79--112.

[37]

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE Transactions on Neural Networks, Vol. 20, 1 (2008), 61--80.

Digital Library

[38]

Yi Shang, Hongchi Shi, and Su-Shing Chen. 2001. An intelligent distributed environment for active learning. Journal on Educational Resources in Computing (JERIC), Vol. 1, 2es (2001), 4--es.

Digital Library

[39]

Shan-Yun Teng, Jundong Li, Lo Pang-Yun Ting, Kun-Ta Chuang, and Huan Liu. 2018. Interactive unknowns recommendation in e-learning systems. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 497--506.

[40]

Shiwei Tong, Qi Liu, Wei Huang, Zhenya Huang, Enhong Chen, Chuanren Liu, Haiping Ma, and Shijin Wang. 2020. Structure-based Knowledge Tracing: An Influence Propagation View. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 541--550.

[41]

Andreas Toscher and Michael Jahrer. 2010. Collaborative filtering applied to educational data mining. KDD cup (2010).

[42]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, 11 (2008).

[43]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[44]

Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, and Shijin Wang. 2020. Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 6153--6161.

[45]

Hao Wang, Tong Xu, Qi Liu, Defu Lian, Enhong Chen, Dongfang Du, Han Wu, and Wen Su. 2019 c. MCNE: An end-to-end framework for learning multiple conditional network representations of social network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1064--1072.

Digital Library

[46]

Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019 b. Heterogeneous graph attention network. In The World Wide Web Conference. 2022--2032.

Digital Library

[47]

Zhiwei Wang, Xiaoqin Feng, Jiliang Tang, Gale Yan Huang, and Zitao Liu. 2019 a. Deep knowledge tracing with side information. In International Conference on AIEd. Springer, 303--308.

Digital Library

[48]

Robert S Woodworth and EL Thorndike. 1901. The influence of improvement in one mental function upon the efficiency of other functions.(I). Psychological review, Vol. 8, 3 (1901), 247.

[49]

Haiqin Yang and Lap Pong Cheung. 2018. Implicit heterogeneous features embedding in deep knowledge tracing. Cognitive Computation, Vol. 10, 1 (2018), 3--14.

[50]

Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. 2019. Graph transformer networks. In Advances in Neural Information Processing Systems. 11983--11993.

[51]

Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogeneous graph neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 793--803.

Digital Library

[52]

Yuhao Zhang, Peng Qi, and Christopher D Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. arXiv preprint arXiv:1809.10185 (2018).

Cited By

Huang QChen J(2024)Enhancing academic performance prediction with temporal graph networks for massive open online coursesJournal of Big Data10.1186/s40537-024-00918-511:1Online publication date: 13-Apr-2024
https://doi.org/10.1186/s40537-024-00918-5
Liu FBu CZhang HWu LYu KHu X(2024)FDKT: Towards an Interpretable Deep Knowledge Tracing via Fuzzy ReasoningACM Transactions on Information Systems10.1145/365616742:5(1-26)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3656167
Sun JMei SYuan KJiang YCao J(2024)Prerequisite-Enhanced Category-Aware Graph Neural Networks for Course RecommendationACM Transactions on Knowledge Discovery from Data10.1145/364364418:5(1-21)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3643644
Show More Cited By

Index Terms

RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems
1. Applied computing
  1. Education
    1. E-learning

Recommendations

Incremental Cognitive Diagnosis for Intelligent Education
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Cognitive diagnosis, aiming at providing an approach to reveal the proficiency level of learners on knowledge concepts, plays an important role in intelligent education area and has recently received more and more attention. Although a number of works ...
Read More
Knowledge-Sensed Cognitive Diagnosis for Intelligent Education Platforms
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Cognitive diagnosis is a fundamental issue of intelligent education platforms, whose goal is to reveal the mastery of students on knowledge concepts. Recently, certain efforts have been made to improve the diagnosis precision, by designing deep neural ...
Read More
Graph-based cognitive diagnosis for intelligent tutoring systems
Abstract
For intelligent tutoring systems, Cognitive Diagnosis (CD) is a fundamental task that aims to estimate the mastery degree of a student on each skill according to the exercise record. The CD task is considered rather challenging since ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Foundation of State Key Laboratory of Cognitive Intelligence
National Natural Science Foundation of China

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
804
Total Downloads

Downloads (Last 12 months)187
Downloads (Last 6 weeks)29

Other Metrics

View Author Metrics

Citations

Cited By

Huang QChen J(2024)Enhancing academic performance prediction with temporal graph networks for massive open online coursesJournal of Big Data10.1186/s40537-024-00918-511:1Online publication date: 13-Apr-2024
https://doi.org/10.1186/s40537-024-00918-5
Liu FBu CZhang HWu LYu KHu X(2024)FDKT: Towards an Interpretable Deep Knowledge Tracing via Fuzzy ReasoningACM Transactions on Information Systems10.1145/365616742:5(1-26)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3656167
Sun JMei SYuan KJiang YCao J(2024)Prerequisite-Enhanced Category-Aware Graph Neural Networks for Course RecommendationACM Transactions on Knowledge Discovery from Data10.1145/364364418:5(1-21)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3643644
Cui CYao YZhang CMa HMa YRen ZZhang CKo J(2024)DGEKT: A Dual Graph Ensemble Learning Method for Knowledge TracingACM Transactions on Information Systems10.1145/363835042:3(1-24)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3638350
Yu XQin CShen DMa HZhang LZhang XZhu HXiong H(2024)RDGT: Enhancing Group Cognitive Diagnosis With Relation-Guided Dual-Side Graph TransformerIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.335264036:7(3429-3442)Online publication date: Jul-2024
https://doi.org/10.1109/TKDE.2024.3352640
Sun JDu SLiu ZYu FLiu SShen X(2024)Weighted Heterogeneous Graph-Based Three-View Contrastive Learning for Knowledge Tracing in Personalized e-Learning SystemsIEEE Transactions on Consumer Electronics10.1109/TCE.2023.329395370:1(2838-2847)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3293953
Yang XSun YYang JCao YZhang AShi WJu JYan QYin JWang Z(2024)Recurrent Neural Cognitive Diagnosis with Semantic Analysis and Self-attention2024 7th International Symposium on Autonomous Systems (ISAS)10.1109/ISAS61044.2024.10552584(1-6)Online publication date: 7-May-2024
https://doi.org/10.1109/ISAS61044.2024.10552584
Chen XFeng SYang MZhao KXu RCui CChen M(2024)Modeling question difficulty for unbiased cognitive diagnosis: A causal perspectiveKnowledge-Based Systems10.1016/j.knosys.2024.111750294(111750)Online publication date: Jun-2024
https://doi.org/10.1016/j.knosys.2024.111750
Su YHan ZShen SYang XHuang ZWu JZhou HLiu Q(2024)Constructing a Confidence-guided Multigraph Model for cognitive diagnosis in personalized learningExpert Systems with Applications10.1016/j.eswa.2024.124259252(124259)Online publication date: Oct-2024
https://doi.org/10.1016/j.eswa.2024.124259
Duan ZDong XGu HWu XLi ZZhou D(2024)Towards more accurate and interpretable modelExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122573243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122573
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents