research-article

Leveraging Transferable Knowledge Concept Graph Embedding for Cold-Start Cognitive Diagnosis

Authors:

Shijin WangAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 983 - 992

https://doi.org/10.1145/3539618.3591774

Published: 18 July 2023 Publication History

Abstract

Cognitive diagnosis (CD) aims to reveal the proficiency of students on specific knowledge concepts and traits of test exercises (e.g., difficulty). It plays a critical role in intelligent education systems by supporting personalized learning guidance. However, recent developments in CD mostly concentrate on improving the accuracy of diagnostic results and often overlook the important and practical task: domain-level zero-shot cognitive diagnosis (DZCD). The primary challenge of DZCD is the deficiency of student behavior data in the target domain due to the absence of student-exercise interactions or unavailability of exercising records for training purposes. To tackle the cold-start issue, we propose a two-stage solution named TechCD (Transferable knowledgE Concept grapH embedding framework for Cognitive Diagnosis). The fundamental notion involves utilizing a pedagogical knowledge concept graph (KCG) as a mediator to connect disparate domains, allowing the transmission of student cognitive signals from established domains to the zero-shot cold-start domain. Specifically, a naive yet effective graph convolutional network (GCN) with the bottom-layer discarding operation is initially employed over the KCG to learn transferable student cognitive states and domain-specific exercise traits. Moreover, we give three implementations of the general TechCD framework following the typical cognitive diagnosis solutions. Finally, extensive experiments on real-world datasets not only prove that Tech can effectively perform zero-shot diagnosis, but also give some popular applications such as exercise recommendation.

Supplemental Material

MP4 File

Presentation video of SIGIR-2023 full paper "Leveraging Transferable Knowledge Concept Graph Embedding for Cold-Start Cognitive Diagnosis".

Download
87.54 MB

References

[1]

Haoyang Bi, Haiping Ma, Zhenya Huang, Yu Yin, Qi Liu, Enhong Chen, Yu Su, and Shijin Wang. 2020. Quality meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 42--51.

[2]

Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. 2015. Modeling Exercise Relationships in E-Learning: A Unified Approach. In EDM. 532--535.

[3]

Penghe Chen, Yu Lu, Vincent W Zheng, and Yang Pian. 2018. Prerequisite-driven deep knowledge tracing. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 39--48.

[4]

Shuo Chen and Thorsten Joachims. 2016. Predicting matchups and preferences in context. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 775--784.

Digital Library

[5]

Jimmy De La Torre. 2009. DINA model and parameter estimation: A didactic. Journal of educational and behavioral statistics, Vol. 34, 1 (2009), 115--130.

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[7]

Nan Ding and Radu Soricut. 2017. Cold-start reinforcement learning with softmax policy gradient. Advances in Neural Information Processing Systems, Vol. 30 (2017).

[8]

Henry C Ellis. 1965. The transfer of learning. (1965).

[9]

Susan E Embretson and Steven P Reise. 2013. Item response theory. Psychology Press.

[10]

Weibo Gao, Qi Liu, Zhenya Huang, Yu Yin, Haoyang Bi, Mu-Chun Wang, Jianhui Ma, Shijin Wang, and Yu Su. 2021. Rcd: Relation map driven cognitive diagnosis for intelligent education systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 501--510.

Digital Library

[11]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth International conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249--256.

[12]

Margaret Grogan. 1999. Equity/equality issues of gender, race, and class. Educational Administration Quarterly, Vol. 35, 4 (1999), 518--536.

[13]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 639--648.

Digital Library

[14]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International conference on world wide web. 173--182.

Digital Library

[15]

Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020b. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems (TOIS), Vol. 38, 3 (2020), 1--32.

Digital Library

[16]

Wenbing Huang, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang. 2020a. Tackling over-smoothing for general graph convolutional networks. arXiv preprint arXiv:2008.09864 (2020).

[17]

Xiaoqing Huang, Qi Liu, Chao Wang, Haoyu Han, Jianhui Ma, Enhong Chen, Yu Su, and Shijin Wang. 2019a. Constructing Educational Concept Maps with Multiple Relationships from Multi-Source Data. In 2019 IEEE ICDM. IEEE, 1108--1113.

[18]

Zhenya Huang, Qi Liu, Chengxiang Zhai, Yu Yin, Enhong Chen, Weibo Gao, and Guoping Hu. 2019b. Exploring multi-objective exercise recommendations in online education systems. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1261--1270.

Digital Library

[19]

Yujia Huo, Derek F Wong, Lionel M Ni, Lidia S Chao, and Jing Zhang. 2020. Knowledge modeling via contextualized representations for LSTM-based personalized exercise recommendation. Information Sciences, Vol. 523 (2020), 266--278.

[20]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[21]

Jiatong Li, Fei Wang, Qi Liu, Mengxiao Zhu, Wei Huang, Zhenya Huang, Enhong Chen, Yu Su, and Shijin Wang. 2022. HierCDF: A Bayesian Network-based Hierarchical Cognitive Diagnosis Framework. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 904--913.

Digital Library

[22]

Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Hui Xiong, Yu Su, and Guoping Hu. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, Vol. 33, 1 (2019), 100--115.

Digital Library

[23]

Qi Liu, Runze Wu, Enhong Chen, Guandong Xu, Yu Su, Zhigang Chen, and Guoping Hu. 2018. Fuzzy cognitive diagnosis for modelling examinee performance. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 9, 4 (2018), 1--26.

Digital Library

[24]

Ye Liu, Han Wu, Zhenya Huang, Hao Wang, Jianhui Ma, Qi Liu, Enhong Chen, Hanqing Tao, and Ke Rui. 2020. Technical phrase extraction for patent mining: A multi-level approach. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 1142--1147.

[25]

Nima Mirbakhsh and Charles X Ling. 2015. Improving top-n recommendation for cold-start users via cross-domain information. ACM Transactions on Knowledge Discovery from Data (TKDD), Vol. 9, 4 (2015), 1--19.

[26]

Hiromi Nakagawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Graph-based Knowledge Tracing: Modeling Student Proficiency Using Graph Neural Network. In 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI). IEEE, 156--163.

[27]

Tuan Nguyen. 2015. The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT Journal of online learning and teaching, Vol. 11, 2 (2015), 309--319.

[28]

Liangming Pan, Chengjiang Li, Juanzi Li, and Jie Tang. 2017. Prerequisite relation learning for concepts in moocs. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1447--1456.

[29]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, Vol. 32 (2019).

[30]

Mark D Reckase. 2009. Multidimensional item response theory models. In Multidimensional item response theory. Springer, 79--112.

[31]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).

[32]

Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. 2002. Methods and metrics for cold-start recommendations. In Proceedings of the 25th annual International ACM SIGIR conference on Research and development in information retrieval. 253--260.

Digital Library

[33]

Robin Schmucker and Tom M Mitchell. 2022. Transferable Student Performance Modeling for Intelligent Tutoring Systems. arXiv preprint arXiv:2202.03980 (2022).

[34]

Yi Shang, Hongchi Shi, and Su-Shing Chen. 2001. An intelligent distributed environment for active learning. Journal on Educational Resources in Computing (JERIC), Vol. 1, 2es (2001), 4--es.

Digital Library

[35]

Yu Su, Qingwen Liu, Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Chris Ding, Si Wei, and Guoping Hu. 2018. Exercise-enhanced sequential modeling for student performance prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[36]

Shan-Yun Teng, Jundong Li, Lo Pang-Yun Ting, Kun-Ta Chuang, and Huan Liu. 2018. Interactive unknowns recommendation in e-learning systems. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 497--506.

[37]

Shiwei Tong, Jiayu Liu, Yuting Hong, Zhenya Huang, Le Wu, Qi Liu, Wei Huang, Enhong Chen, and Dan Zhang. 2022. Incremental Cognitive Diagnosis for Intelligent Education. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1760--1770.

Digital Library

[38]

Emiko Tsutsumi, Ryo Kinoshita, and Maomi Ueno. 2021. Deep-IRT with Independent Student and Item Networks. International Educational Data Mining Society (2021).

[39]

Kurt VanLehn. 2011. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational psychologist, Vol. 46, 4 (2011), 197--221.

[40]

Manasi Vartak, Arvind Thiagarajan, Conrado Miranda, Jeshua Bratman, and Hugo Larochelle. 2017. A meta-learning perspective on cold-start recommendations for items. Advances in neural information processing systems, Vol. 30 (2017).

[41]

Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, and Shijin Wang. 2020. Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 6153--6161.

[42]

Hao Wang, Enhong Chen, Qi Liu, Tong Xu, Dongfang Du, Wen Su, and Xiaopeng Zhang. 2018. A united approach to learning sparse attributed network embedding. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 557--566.

[43]

Hao Wang, Defu Lian, Hanghang Tong, Qi Liu, Zhenya Huang, and Enhong Chen. 2021. Hypersorec: Exploiting hyperbolic user and item representations with multiple aspects for social-aware recommendation. ACM Transactions on Information Systems (TOIS), Vol. 40, 2 (2021), 1--28.

Digital Library

[44]

Hao Wang, Tong Xu, Qi Liu, Defu Lian, Enhong Chen, Dongfang Du, Han Wu, and Wen Su. 2019a. MCNE: An end-to-end framework for learning multiple conditional network representations of social network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1064--1072.

Digital Library

[45]

Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. 2019b. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 (2019).

[46]

Zhengyang Wu, Ming Li, Yong Tang, and Qingyu Liang. 2020. Exercise recommendation based on knowledge concept prediction. Knowledge-Based Systems, Vol. 210 (2020), 106481.

[47]

Jie Xu, Cheng Deng, Xinbo Gao, Dinggang Shen, and Heng Huang. 2017. Predicting Alzheimer's disease cognitive assessment via robust low-rank structured sparse model. In IJCAI: proceedings of the conference, Vol. 2017. NIH Public Access, 3880.

[48]

Linan Yue, Qi Liu, Yichao Du, Yanqing An, Li Wang, and Enhong Chen. 2022. DARE: Disentanglement-Augmented Rationale Extraction. Advances in Neural Information Processing Systems, Vol. 35 (2022), 26603--26617.

[49]

Si Zhang, Hanghang Tong, Jiejun Xu, and Ross Maciejewski. 2019. Graph convolutional networks: a comprehensive review. Computational Social Networks, Vol. 6, 1 (2019), 1--23.

[50]

Hao Zhao, Ming Lu, Anbang Yao, Yurong Chen, and Li Zhang. 2020a. Learning to draw sight lines. International Journal of Computer Vision, Vol. 128 (2020), 1076--1100.

Digital Library

[51]

Hao Zhao, Ming Lu, Anbang Yao, Yiwen Guo, Yurong Chen, and Li Zhang. 2017. Physics inspired optimization on semantic transfer features: An alternative method for room layout estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 10--18.

[52]

Hao Zhao, Ming Lu, Anbang Yao, Yiwen Guo, Yurong Chen, and Li Zhang. 2020b. Pointly-supervised scene parsing with uncertainty mixture. Computer Vision and Image Understanding, Vol. 200 (2020), 103040.

[53]

Wayne Xin Zhao, Wenhui Zhang, Yulan He, Xing Xie, and Ji-Rong Wen. 2018. Automatically learning topics and difficulty levels of problems in online judge systems. ACM Transactions on Information Systems (TOIS), Vol. 36, 3 (2018), 1--33.

Digital Library

[54]

Jianhuan Zhuo, Jianxun Lian, Lanling Xu, Ming Gong, Linjun Shou, Daxin Jiang, Xing Xie, and Yinliang Yue. 2022. Tiger: Transferable Interest Graph Embedding for Domain-Level Zero-Shot Recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2806--2816.

Digital Library

Cited By

Zhang DZhang KWu LTian MHong RWang MBaeza-Yates RBonchi F(2024)Path-Specific Causal Reasoning for Fairness-aware Cognitive DiagnosisProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672049(4143-4154)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672049
Cui JQian HJiang BZhang WBaeza-Yates RBonchi F(2024)Leveraging Pedagogical Theories to Understand Student Learning Process with Graph-based Reasonable Knowledge TracingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671853(502-513)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671853
Yao FLiu QYue LGao WLi JLi XHe YBaeza-Yates RBonchi F(2024)AdaRD: An Adaptive Response Denoising Framework for Robust Learner ModelingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671684(3886-3895)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671684
Show More Cited By

Index Terms

Leveraging Transferable Knowledge Concept Graph Embedding for Cold-Start Cognitive Diagnosis
1. Applied computing
  1. Education
    1. E-learning

Recommendations

Personalization Exercise Recommendation Based on Cognitive Diagnosis
CSAE '22: Proceedings of the 6th International Conference on Computer Science and Application Engineering

Online education has become an important part of education services. Faced with the vast online learning resources, how to provide students with personalized learning resources and achieve the goal of precise teaching is an important part of the ...
Improving Cold Start Recommendation by Mapping Feature-Based Preferences to Item Comparisons
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

Many Recommender Systems (RSs) rely on user preference data in the form of ratings or likes for items. Previous research has shown that item comparisons can also be effectively used to model user preferences and build RS. However, users often express ...
Item cold-start recommendations: learning local collective embeddings
RecSys '14: Proceedings of the 8th ACM Conference on Recommender systems

Recommender systems suggest to users items that they might like (e.g., news articles, songs, movies) and, in doing so, they help users deal with information overload and enjoy a personalized experience. One of the main problems of these systems is the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Program of China
National Natural Science Foundation of China
Open Research Fund of the State Key Laboratory of Cognitive Intelligence

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
553
Total Downloads

Downloads (Last 12 months)345
Downloads (Last 6 weeks)43

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang DZhang KWu LTian MHong RWang MBaeza-Yates RBonchi F(2024)Path-Specific Causal Reasoning for Fairness-aware Cognitive DiagnosisProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672049(4143-4154)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672049
Cui JQian HJiang BZhang WBaeza-Yates RBonchi F(2024)Leveraging Pedagogical Theories to Understand Student Learning Process with Graph-based Reasonable Knowledge TracingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671853(502-513)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671853
Yao FLiu QYue LGao WLi JLi XHe YBaeza-Yates RBonchi F(2024)AdaRD: An Adaptive Response Denoising Framework for Robust Learner ModelingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671684(3886-3895)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671684
Gan CHuang BHu BMa JZhang ZZhou JZhang GZhong WAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)PEACE: Prototype lEarning Augmented transferable framework for Cross-domain rEcommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635781(228-237)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635781
Zhang ZLiu QHu ZZhan YHuang ZGao WMao QChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Enhancing Fairness in Meta-learned User Modeling via Adaptive SamplingProceedings of the ACM Web Conference 202410.1145/3589334.3645369(3241-3252)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645369
Yue LLiu QLiu YGao WYao FLi WChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Cooperative Classification and Rationalization for Graph GeneralizationProceedings of the ACM Web Conference 202410.1145/3589334.3645332(344-352)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645332
Liu DLin XLi LMing Z(2024)Teaching content recommendations in music appreciation courses via graph embedding learningInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02123-515:9(3847-3862)Online publication date: 16-May-2024
https://doi.org/10.1007/s13042-024-02123-5
Zhang ZWu LLiu QLiu JHuang ZYin YZhuang YGao WChen E(2024)Understanding and improving fairness in cognitive diagnosisScience China Information Sciences10.1007/s11432-022-3852-067:5Online publication date: 25-Apr-2024
https://doi.org/10.1007/s11432-022-3852-0
Jin XLiu QYue LLiu YZhao LGao WGong ZZhang KBi H(2024)Diagnosis Then Aggregation: An Adaptive Ensemble Strategy for Keyphrase ExtractionArtificial Intelligence10.1007/978-981-99-8850-1_46(566-578)Online publication date: 4-Feb-2024
https://doi.org/10.1007/978-981-99-8850-1_46
Peng JLiu QYue LZhang ZZhang KSha Y(2024)Towards Few-Shot Self-explaining Graph Neural NetworksMachine Learning and Knowledge Discovery in Databases. Research Track10.1007/978-3-031-70365-2_7(109-126)Online publication date: 22-Aug-2024
https://doi.org/10.1007/978-3-031-70365-2_7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents