Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Multi-objective Test Recommendation for Adaptive Learning

  • Chapter
  • First Online:
Transactions on Large-Scale Data- and Knowledge-Centered Systems LVI

Part of the book series: Lecture Notes in Computer Science ((TLDKS,volume 14790))

  • 116 Accesses

Abstract

Upskilling is a fast-growing segment of the education economy [31]. Yet, there is little algorithmic work that focuses on crafting dedicated strategies to reach high-skill mastery. In this paper, we formalize AdUp, an iterative upskilling problem that combines mastery learning [49] and Zone of Proximal Development [7]. We extend our previous work [9] and design two solutions for AdUp: MOO and MAB. MOO is a multi-objective optimization approach that relies on Hill Climbing to adapt the difficulty of recommended tests to three objectives: learner’s predicted performance, aptitude, and skill gap. MAB is a meta approach based on Multi-Armed Bandits to learn the best combination of objectives to optimize at each iteration. We show how these solutions are combined with two common learner simulation models: BKT (KT-IDEM) [47] and Item Response Theory (IRT) [53]. Our simulation experiments demonstrate the necessity of leveraging all three objectives and the need to adapt the optimization objectives to the learner’s progression ability as MAB offers a higher mastery rate and a better final skill gain than MOO.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/adaptive-learning/matmat-web/blob/master/data/data_description.md.

  2. 2.

    https://sites.google.com/view/assistmentsdatamining.

  3. 3.

    https://github.com/adaptive-learning/matmat-web/blob/master/data/data_description.md.

  4. 4.

    https://sites.google.com/view/assistmentsdatamining.

  5. 5.

    https://new.assistments.org.

  6. 6.

    https://sites.google.com/site/assistmentsdata/home/2009-2010-assistment-data/skill-builder-data-2009-2010?authuser=0.

  7. 7.

    https://github.com/hcnoh/knowledge-tracing-collection-pytorch/tree/main.

  8. 8.

    https://github.com/AdaptiveUpskilling/AdUp.git.

References

  1. Abdelrahman, G., Wang, Q.: Knowledge tracing with sequential key-value memory networks. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 175–184 (2019)

    Google Scholar 

  2. Abdelrahman, G., Wang, Q., Nunes, B.: Knowledge tracing: a survey. ACM Comput. Surv. 55(11), 1–37 (2023)

    Article  Google Scholar 

  3. Abdi, S., Khosravi, H., Sadiq, S., Darvishi, A.: Open learner models for multi-activity educational systems. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds.) AIED 2021, Part II. LNCS (LNAI), vol. 12749, pp. 11–17. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78270-2_2

    Chapter  Google Scholar 

  4. Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: a brief survey. IEEE Sig. Process. Mag. 34(6), 26–38 (2017)

    Article  Google Scholar 

  5. Badrinath, A., Wang, F., Pardos, Z.: pyBKT: an accessible python library of Bayesian knowledge tracing models. In: Proceedings of the 14th International Conference on Educational Data Mining, pp. 468–474 (2021)

    Google Scholar 

  6. Bartolini, I., Ciaccia, P., Patella, M.: Efficient sort-based skyline evaluation. ACM Trans. Database Syst. (TODS) 33(4), 1–49 (2008)

    Article  Google Scholar 

  7. Basawapatna, A.R., Repenning, A., Koh, K.H., Nickerson, H.: The zones of proximal flow: guiding students through a space of computational thinking skills and challenges. In: Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research, pp. 67–74 (2013)

    Google Scholar 

  8. Birnbaum, A.: Some latent trait models and their use in inferring an examinee’s ability. In: Statistical Theories of Mental Test Scores (1968)

    Google Scholar 

  9. Bouarour, N., Benouaret, I., D’Ham, C., Amer-Yahia, S.: Adaptive test recommendation for mastery learning. In: Proceedings of the 2nd International Workshop on Data Systems Education: Bridging Education Practice with Education Research, pp. 18–23 (2023)

    Google Scholar 

  10. Bruffee, K.A.: Collaborative Learning: Higher Education, Interdependence, and the Authority of Knowledge. ERIC (1999)

    Google Scholar 

  11. Cen, H., Koedinger, K., Junker, B.: Learning factors analysis – a general method for cognitive model evaluation and improvement. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 164–175. Springer, Heidelberg (2006). https://doi.org/10.1007/11774303_17

    Chapter  Google Scholar 

  12. Cen, H., Koedinger, K., Junker, B.: Comparing two IRT models for conjunctive skills. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 796–798. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69132-7_111

    Chapter  Google Scholar 

  13. Coetzee, D., Lim, S., Fox, A., Hartmann, B., Hearst, M.A.: Structuring interactions for large-scale synchronous peer learning. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, pp. 1139–1152 (2015)

    Google Scholar 

  14. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User Adap. Inter. 4(4), 253–278 (1994)

    Article  Google Scholar 

  15. Csikszentmihalyi, M.: Beyond Boredom and Anxiety: The Experience of Play in Work and Games. Jossey-Bass (1975)

    Google Scholar 

  16. David, Y.B., Segal, A., Gal, Y.: Sequencing educational content in classrooms using Bayesian knowledge tracing. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 354–363 (2016)

    Google Scholar 

  17. Davis, D., Chen, G., Hauff, C., Houben, G.J.: Activating learning at scale: a review of innovations in online learning strategies. Comput. Educ. 125, 327–344 (2018)

    Article  Google Scholar 

  18. De Vin, L.J., Jacobsson, L., Odhe, J., Wickberg, A.: Lean production training for the manufacturing industry: experiences from Karlstad Lean Factory. Procedia Manuf. 11, 1019–1026 (2017)

    Article  Google Scholar 

  19. Esfandiari, M., Wei, D., Amer-Yahia, S., Basu Roy, S.: Optimizing peer learning in online groups with affinities. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1216–1226 (2019)

    Google Scholar 

  20. Fischer, C., et al.: Mining big data in education: affordances and challenges. Rev. Res. Educ. 44(1), 130–160 (2020)

    Article  Google Scholar 

  21. Gadiraju, U., Dietze, S.: Improving learning through achievement priming in crowdsourced information finding microtasks. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 105–114. ACM (2017)

    Google Scholar 

  22. Gašević, D., Kovanović, V., Joksimović, S., Siemens, G.: Where is research on massive open online courses headed? A data analysis of the MOOC research initiative. Int. Rev. Res. Open Distrib. Learn. 15(5), 134–176 (2014)

    Google Scholar 

  23. Gong, Y., Beck, J.E., Heffernan, N.T.: Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010. LNCS, vol. 6094, pp. 35–44. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13388-6_8

    Chapter  Google Scholar 

  24. Heffernan, N.T., Heffernan, C.L.: The assistments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. Int. J. Artif. Intell. Educ. 24(4), 470–497 (2014)

    Article  MathSciNet  Google Scholar 

  25. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  26. Huang, Z., et al.: Exploring multi-objective exercise recommendations in online education systems. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1261–1270 (2019)

    Google Scholar 

  27. Kelly, K., Wang, Y., Thompson, T., Heffernan, N.: Defining mastery: knowledge tracing versus n-consecutive correct responses. In: Student Modeling From Different Aspects, p. 39 (2016)

    Google Scholar 

  28. Kolb, D.A.: Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs (1984)

    Google Scholar 

  29. Lave, J., Wenger, E.: Situated Learning: Legitimate Peripheral Participation. Cambridge University Press (1991)

    Google Scholar 

  30. Lee, Y.W., Sawaki, Y.: Cognitive diagnosis approaches to language assessment: an overview. Lang. Assess. Q. 6(3), 172–189 (2009)

    Article  Google Scholar 

  31. Li, L.: Reskilling and upskilling the future-ready workforce for industry 4.0 and beyond. Inf. Syst. Front., 1–16 (2022). https://doi.org/10.1007/s10796-022-10308-y

  32. Lipton, Z.C., Berkowitz, J., Elkan, C.: A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019 (2015)

  33. Liu, Q., et al.: Exploiting cognitive structure for adaptive learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 627–635 (2019)

    Google Scholar 

  34. Lord, F.M.: Applications of Item Response Theory to Practical Testing Problems. Routledge (1980)

    Google Scholar 

  35. Mao, Y., et al.: Learning behavior-aware cognitive diagnosis for online education systems. In: Zeng, J., Qin, P., Jing, W., Song, X., Lu, Z. (eds.) ICPCSEE 2021. CCIS, vol. 1452, pp. 385–398. Springer, Singapore (2021). https://doi.org/10.1007/978-981-16-5943-0_31

    Chapter  Google Scholar 

  36. Matsubara, M., Borromeo, R.M., Amer-Yahia, S., Morishima, A.: Task assignment strategies for crowd worker ability improvement. Proc. ACM Hum. Comput. Interact. 5(CSCW2), 1–20 (2021)

    Article  Google Scholar 

  37. Miller, A., Fisch, A., Dodge, J., Karimi, A.H., Bordes, A., Weston, J.: Key-value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126 (2016)

  38. Mincer, J.: On-the-job training: costs, returns, and some implications. J. Polit. Econ. 70(5, Part 2), 50–79 (1962)

    Article  Google Scholar 

  39. Minn, S.: BKT-LSTM: efficient student modeling for knowledge tracing and student performance prediction. arXiv preprint arXiv:2012.12218 (2020)

  40. Mujtaba, D.F., Mahapatra, N.R.: Multi-objective optimization of item selection in computerized adaptive testing. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1018–1026 (2021)

    Google Scholar 

  41. Nakagawa, H., Iwasawa, Y., Matsuo, Y.: Graph-based knowledge tracing: modeling student proficiency using graph neural network. In: IEEE/WIC/ACM International Conference on Web Intelligence, pp. 156–163 (2019)

    Google Scholar 

  42. Omidvar-Tehrani, B., Amer-Yahia, S., Dutot, P.-F., Trystram, D.: Multi-objective group discovery on the social web. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J. (eds.) ECML PKDD 2016. LNCS (LNAI), vol. 9851, pp. 296–312. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46128-1_19

    Chapter  Google Scholar 

  43. Ostrow, K., Donnelly, C., Adjei, S., Heffernan, N.: Improving student modeling through partial credit and problem difficulty. In: Proceedings of the Second 2015 ACM Conference on Learning@ Scale, pp. 11–20 (2015)

    Google Scholar 

  44. Pandey, S., Karypis, G.: A self-attentive model for knowledge tracing. arXiv preprint arXiv:1907.06837 (2019)

  45. Panel, H.E.S.: Final report–improving retention, completion and success in higher education (2017)

    Google Scholar 

  46. Papadimitriou, C.H., Yannakakis, M.: On the approximability of trade-offs and optimal access of web sources. In: Proceedings 41st Annual Symposium on Foundations of Computer Science, pp. 86–92. IEEE (2000)

    Google Scholar 

  47. Pardos, Z.A., Heffernan, N.T.: KT-IDEM: introducing item difficulty to the knowledge tracing model. In: Konstan, J.A., Conejo, R., Marzo, J.L., Oliver, N. (eds.) UMAP 2011. LNCS, vol. 6787, pp. 243–254. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22362-4_21

    Chapter  Google Scholar 

  48. Pavlik, P.I., Jr., Cen, H., Koedinger, K.R.: Performance factors analysis–a new alternative to knowledge tracing. In: AIED 2009 (2009)

    Google Scholar 

  49. Pelánek, R., Řihák, J.: Experimental analysis of mastery learning criteria. In: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pp. 156–163 (2017)

    Google Scholar 

  50. Pelánek, R.: Application of time decay functions and Elo system in student modeling. In: Proceedings of the Educational Data Mining, January 2014, pp. 21–27 (2014)

    Google Scholar 

  51. Piech, C., et al.: Deep knowledge tracing. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  52. Rasch, G.: Probabilistic Models for Some Intelligence and Attainment Tests. ERIC (1993)

    Google Scholar 

  53. Reckase, M.D.: Unidimensional item response theory models. In: Multidimensional Item Response Theory. SSBS. Springer, New York (2009). https://doi.org/10.1007/978-0-387-89976-3_2

  54. Rollinson, J., Brunskill, E.: From predictive models to instructional policies. International Educational Data Mining Society (2015)

    Google Scholar 

  55. Rozenblit, L., Keil, F.: The misunderstood limits of folk science: an illusion of explanatory depth. Cogn. Sci. 26(5), 521–562 (2002)

    Article  Google Scholar 

  56. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285–295 (2001)

    Google Scholar 

  57. Su, Y., et al.: Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  58. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (2018)

    Google Scholar 

  59. bigdata ustc: EduCDM (2021). https://github.com/bigdata-ustc/EduCDM

  60. VanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46(4), 197–221 (2011)

    Article  Google Scholar 

  61. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  62. Vygotsky, L.: Zone of proximal development. In: Mind in Society: The Development of Higher Psychological Processes, vol. 5291, p. 157 (1987)

    Google Scholar 

  63. Wang, F., et al.: Neural cognitive diagnosis for intelligent education systems. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 6153–6161 (2020)

    Google Scholar 

  64. Wu, A.S., Farrell, R., Singley, M.K.: Scaffolding group learning in a collaborative networked environment (2002)

    Google Scholar 

  65. Xu, Z., Wijekumar, K., Ramirez, G., Hu, X., Irey, R.: The effectiveness of intelligent tutoring systems on K-12 students’ reading comprehension: a meta-analysis. Br. J. Edu. Technol. 50(6), 3119–3137 (2019)

    Article  Google Scholar 

  66. Zhang, J., Shi, X., King, I., Yeung, D.Y.: Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th International Conference on World Wide Web, pp. 765–774 (2017)

    Google Scholar 

  67. Zhu, H., et al.: A multi-constraint learning path recommendation algorithm based on knowledge map. Knowl. Based Syst. 143, 102–114 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CNRS, Université Grenoble Alpes, Saint-Martin-d’Hères, France

    Nassim Bouarour & Sihem Amer-Yahia

  2. EPITA, Lyon, France

    Idir Benouaret

Authors
  1. Nassim Bouarour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Idir Benouaret
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sihem Amer-Yahia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nassim Bouarour .

Editor information

Editors and Affiliations

  1. IRIT, Paul Sabatier University, Toulouse, France

    Abdelkader Hameurlain

  2. Technical University of Vienna, Vienna, Austria

    A Min Tjoa

  3. Inria and LIRMM, Montpellier, France

    Reza Akbarinia

  4. Claude Bernard Lyon 1 University and Institut Universitaire de France, Villeurbanne, France

    Angela Bonifati

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer-Verlag GmbH, DE, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bouarour, N., Benouaret, I., Amer-Yahia, S. (2024). Multi-objective Test Recommendation for Adaptive Learning. In: Hameurlain, A., Tjoa, A.M., Akbarinia, R., Bonifati, A. (eds) Transactions on Large-Scale Data- and Knowledge-Centered Systems LVI. Lecture Notes in Computer Science(), vol 14790. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-69603-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-69603-3_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-69602-6

  • Online ISBN: 978-3-662-69603-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation