Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3589334.3645437acmconferencesArticle/Chapter ViewAbstractPublication PageswebconfConference Proceedingsconference-collections
research-article

Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm

Published: 13 May 2024 Publication History
  • Get Citation Alerts
  • Abstract

    Personalized learner modeling using cognitive diagnosis (CD), which aims to model learners' cognitive states by diagnosing learner traits from behavioral data, is a fundamental yet significant task in many web learning services. Existing cognitive diagnosis models (CDMs) follow theproficiency-response paradigm that views learner traits and question parameters as trainable embeddings and learns them through learner performance prediction. However, we notice that this paradigm leads to the inevitable non-identifiability and explainability overfitting problem, which is harmful to the quantification of learners' cognitive states and the quality of web learning services. To address these problems, we propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novelresponse-proficiency-response paradigm inspired by encoder-decoder models. Specifically, we first devise the diagnostic module of ID-CDF, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting. Next, we propose a flexible predictive module for ID-CDF to ensure diagnosis preciseness. We further present an implementation of ID-CDF, i.e., ID-CDM, to illustrate its usability. Extensive experiments on four real-world datasets with different characteristics demonstrate that ID-CDF can effectively address the problems without loss of diagnosis preciseness. Our code is available at https://github.com/CSLiJT/ID-CDF.

    Supplemental Material

    MP4 File
    Supplemental video

    References

    [1]
    Markus Bayer, Marc-André Kaufhold, and Christian Reuter. 2023. A Survey on Data Augmentation for Text Classification. ACM Comput. Surv., Vol. 55, 7 (2023), 146:1--146:39. https://doi.org/10.1145/3544558
    [2]
    George Casella Berger, Roger. 2024. Statistical Inference 2 ed.). Chapman and Hall/CRC, New York.
    [3]
    Justyna Brzezinska. 2020. Item response theory models in the measurement theory. Commun. Stat. Simul. Comput., Vol. 49, 12 (2020), 3299--3313.
    [4]
    Lei Chen, Le Wu, Kun Zhang, Richang Hong, Defu Lian, Zhiqiang Zhang, Jun Zhou, and Meng Wang. 2023. Improving Recommendation Fairness via Data Augmentation. In Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW '23). Association for Computing Machinery, New York, NY, USA, 1012--1020. https://doi.org/10.1145/3543507.3583341
    [5]
    Kyunghyun Cho, Bart van Merrienboer, cC aglar Gü lcc ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP. ACL, 1724--1734.
    [6]
    Susan Craw. 2010. Manhattan Distance. Springer US, Boston, MA, 639--639. https://doi.org/10.1007/978-0--387--30164--8_506
    [7]
    Jimmy de la Torre. 2009. DINA Model and Parameter Estimation: A Didactic. Journal of Educational and Behavioral Statistics, Vol. 34, 1 (2009), 115--130.
    [8]
    Mingyu Feng, Neil T. Heffernan, and Kenneth R. Koedinger. 2009. Addressing the assessment challenge with an online system that tutors as it assesses. User Model. User Adapt. Interact., Vol. 19, 3 (2009), 243--266.
    [9]
    Gerhard H. Fischer. 1995. Derivations of the Rasch Model. Springer New York, New York, NY, 15--38.
    [10]
    Francc ois Fouss, Alain Pirotte, Jean-Michel Renders, and Marco Saerens. 2007. Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation. IEEE Trans. Knowl. Data Eng., Vol. 19, 3 (2007), 355--369.
    [11]
    Yingqiang Ge, Juntao Tan, Yan Zhu, Yinglong Xia, Jiebo Luo, Shuchang Liu, Zuohui Fu, Shijie Geng, Zelong Li, and Yongfeng Zhang. 2022. Explainable Fairness in Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR '22). Association for Computing Machinery, New York, NY, USA, 681--691. https://doi.org/10.1145/3477495.3531973
    [12]
    Alan E. Gelfand and Adrian F. M. Smith. 1990. Sampling-Based Approaches to Calculating Marginal Densities. J. Amer. Statist. Assoc., Vol. 85, 410 (1990), 398--409.
    [13]
    Golnaz Ghiasi, Yin Cui, Aravind Srinivas, Rui Qian, Tsung-Yi Lin, Ekin D. Cubuk, Quoc V. Le, and Barret Zoph. 2021. Simple Copy-Paste Is a Strong Data Augmentation Method for Instance Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19--25, 2021. Computer Vision Foundation / IEEE, 2918--2928. https://doi.org/10.1109/CVPR46437.2021.00294
    [14]
    Mark Gierl and Jacqueline Leighton (Eds.). 2007. Cognitive Diagnostic Assessment for Education: Theory and Applications. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511611186
    [15]
    Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS (JMLR Proceedings, Vol. 9). JMLR.org, 249--256.
    [16]
    W. K. Hastings. 1970. Monte Carlo Sampling Methods Using Markov Chains and Their Applications. Biometrika, Vol. 57, 1 (1970), 97--109.
    [17]
    Stamper J., Niculescu-Mizil A., Ritter S., G.J. Gordon, and Koedinger K.R. 2010. Algebra | 2006--2007. Development data set from KDD Cup 2010 Educational Data Mining Challenge. (2010). http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp
    [18]
    Taegwan Kang, Hwanhee Lee, Byeongjin Choe, and Kyomin Jung. 2021. Entangled Bidirectional Encoder to Autoregressive Decoder for Sequential Recommendation. In SIGIR. ACM, 1657--1661.
    [19]
    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR (Poster).
    [20]
    Sheng Li, Quanlong Guan, Liangda Fang, Fang Xiao, Zhenyu He, Yizhou He, and Weiqi Luo. 2022. Cognitive Diagnosis Focusing on Knowledge Concepts. In CIKM. ACM, 3272--3281.
    [21]
    Xiaopeng Li and James She. 2017. Collaborative Variational Autoencoder for Recommender Systems. In KDD. ACM, 305--314.
    [22]
    Qi Liu. 2021. Towards a New Generation of Cognitive Diagnosis. In IJCAI. ijcai.org, 4961--4964.
    [23]
    Qi Liu, Run-ze Wu, Enhong Chen, Guandong Xu, Yu Su, Zhigang Chen, and Guoping Hu. 2018. Fuzzy Cognitive Diagnosis for Modelling Examinee Performance. ACM Trans. Intell. Syst. Technol., Vol. 9, 4 (2018), 48:1--48:26.
    [24]
    Jinwei Luo, Mingkai He, Weike Pan, and Zhong Ming. 2023. BGNN: Behavior-aware graph neural network for heterogeneous session-based recommendation. Frontiers of Computer Science, Vol. 17, 5 (12 Jan 2023), 175336. https://doi.org/10.1007/s11704-022--2100-y
    [25]
    Leland McInnes, John Healy, and James Melville. 2020. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arxiv: 1802.03426 [stat.ML]
    [26]
    Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, and Jintao Li. 2021. MDFEND: Multi-domain Fake News Detection. In CIKM. ACM, 3343--3347.
    [27]
    Radek Pelánek. 2017. Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Modeling and User-Adapted Interaction, Vol. 27, 3 (Dec. 2017), 313--350. https://doi.org/10.1007/s11257-017--9193--2
    [28]
    Shaina Raza and Chen Ding. 2022. Fake news detection based on news content and social contexts: a transformer-based approach. Int. J. Data Sci. Anal., Vol. 13, 4 (2022), 335--362.
    [29]
    Mark D. Reckase. 2009. Multidimensional Item Response Theory Models. Springer New York, New York, NY, 79--112.
    [30]
    Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. AutoRec: Autoencoders Meet Collaborative Filtering. In WWW (Companion Volume). ACM, 111--112.
    [31]
    Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., Vol. 15, 1 (2014), 1929--1958.
    [32]
    Kikumi K. Tatsuoka. 1983. Rule Space: An Approach for Dealing with Misconceptions Based on Item Response Theory. Journal of Educational Measurement, Vol. 20, 4 (1983), 345--354.
    [33]
    Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, Shijin Wang, and Yu Su. 2022. NeuralCD: A General Framework for Cognitive Diagnosis. IEEE Transactions on Knowledge and Data Engineering (2022), 1--16.
    [34]
    Jinze Wu, Qi Liu, Zhenya Huang, Yuting Ning, Hao Wang, Enhong Chen, Jinfeng Yi, and Bowen Zhou. 2021. Hierarchical Personalized Federated Learning for User Modeling. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW '21). Association for Computing Machinery, New York, NY, USA, 957--968. https://doi.org/10.1145/3442381.3449926
    [35]
    Lianwei Wu, Yuan Rao, Cong Zhang, Yongqiang Zhao, and Ambreen Nazir. 2023. Category-Controlled Encoder-Decoder for Fake News Detection. IEEE Trans. Knowl. Data Eng., Vol. 35, 2 (2023), 1242--1257.
    [36]
    Mike Wu, Richard Lee Davis, Benjamin W. Domingue, Chris Piech, and Noah D. Goodman. 2020. Variational Item Response Theory: Fast, Accurate, and Expressive. In EDM. International Educational Data Mining Society.
    [37]
    Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative Denoising Auto-Encoders for Top-N Recommender Systems. In WSDM. ACM, 153--162.
    [38]
    Gongjun Xu. 2019. Identifiability and Cognitive Diagnosis Models. Springer International Publishing, Cham, 333--357.
    [39]
    Gongjun Xu and Stephanie Zhang. 2016. Identifiability of Diagnostic Classification Models. Psychometrika, Vol. 81, 3 (Sept. 2016), 625--649. https://doi.org/10.1007/s11336-015--9471-z
    [40]
    Peng Xu and Michel C. Desmarais. 2018. An Empirical Research on Identifiability and Q-matrix Design for DINA model. In EDM. International Educational Data Mining Society (IEDMS).
    [41]
    Chun-Kit Yeung. 2019. Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory. In EDM. International Educational Data Mining Society (IEDMS).
    [42]
    Shengjun Yin, Kailai Yang, and Hongzhi Wang. 2020. A MOOC Courses Recommendation System Based on Learning Behaviours. In ACM TUR-C'20: ACM Turing Celebration Conference, Hefei, China, May 22--24, 2020. ACM, 133--137. https://doi.org/10.1145/3393527.3393550
    [43]
    Jifan Yu, Yuquan Wang, Qingyang Zhong, Gan Luo, Yiming Mao, Kai Sun, Wenzheng Feng, Wei Xu, Shulin Cao, Kaisheng Zeng, Zijun Yao, Lei Hou, Yankai Lin, Peng Li, Jie Zhou, Bin Xu, Juanzi Li, Jie Tang, and Maosong Sun. 2021. MOOCCubeX: A Large Knowledge-Centered Repository for Adaptive Learning in MOOCs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM '21). Association for Computing Machinery, New York, NY, USA, 4643--4652. https://doi.org/10.1145/3459637.3482010
    [44]
    Chuang Zhao, Hongke Zhao, Ming HE, Jian Zhang, and Jianping Fan. 2023. Cross-domain recommendation via user interest alignment. In Proceedings of the ACM Web Conference 2023 (Austin, TX, USA) (WWW '23). Association for Computing Machinery, New York, NY, USA, 887--896.

    Index Terms

    1. Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        WWW '24: Proceedings of the ACM on Web Conference 2024
        May 2024
        4826 pages
        ISBN:9798400701719
        DOI:10.1145/3589334
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 13 May 2024

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. cognitive diagnosis
        2. explainability
        3. identifiability
        4. intelligent education
        5. user modeling

        Qualifiers

        • Research-article

        Funding Sources

        Conference

        WWW '24
        Sponsor:
        WWW '24: The ACM Web Conference 2024
        May 13 - 17, 2024
        Singapore, Singapore

        Acceptance Rates

        Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 75
          Total Downloads
        • Downloads (Last 12 months)75
        • Downloads (Last 6 weeks)16
        Reflects downloads up to 12 Aug 2024

        Other Metrics

        Citations

        View Options

        Get Access

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media