Abstract
In online learning platforms such as MOOCs, computerized assessment needs to be optimized in order to prevent boredom and dropout of learners. Indeed, they should spend as little time as possible in tests and still receive valuable feedback. It is actually possible to reduce the number of questions for the same accuracy with computerized adaptive testing (CAT): asking the next question according to the past performance of the examinee. CAT algorithms are divided in two categories: summative CATs, that measure the level of examinees, and formative CATs, that provide feedback to the examinees at the end of the test by specifying which knowledge components need further work. In this paper, we formalize the problem of test-size reduction by predicting student performance, and propose a new hybrid CAT algorithm GenMA based on the general diagnostic model, that is both summative and formative. Using real datasets, we compare our model to popular CAT models and show that GenMA achieves better accuracy while using fewer questions than the existing models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zernike, K.: Obama administration calls for limits on testing in schools (2015)
Council, G.M.A.: Profile of GMAT Candidates - Executive Summary (2013)
Huebner, A.: An overview of recent developments in cognitive diagnostic computer adaptive assessments. Pract. Assess. Res. Eval. 15(3), n3 (2010)
Yan, D., von Davier, A.A., Lewis, C.: Computerized multistage testing (2014)
Lan, A.S., Waters, A.E., Studer, C., Baraniuk, R.G.: Sparse factor analysis for learning and content analytics. J. Mach. Learn. Res. 15, 1959–2008 (2014)
Davier, M.: A general diagnostic model applied to language testing data. ETS Research Report Series, p. i-35 (2005)
Desmarais, M.C., Baker, R.S.: A review of recent advances in learner and skill modeling in intelligent learning environments. User Model. User Adap. Inter. 22, 9–38 (2012)
Bergner, Y., Droschler, S., Kortemeyer, G., Rayyan, S., Seaton, D., Pritchard, D.E.: Model-based collaborative filtering analysis of student response data: machine-learning item response theory. In: International Educational Data Mining Society (2012)
Verhelst, N.D.: Profile analysis: a closer look at the PISA 2000 reading data. Scand. J. Educ. Res. 56, 315–332 (2012)
Reckase, M.: Multidimensional Item Response Theory. Statistics for Social and Behavioral Sciences. Springer, New York (2009)
Xu, X., Chang, H., Douglas, J.: A simulation study to compare CAT strategies for cognitive diagnosis. In: Annual Meeting of the American Educational Research Association, Chicago (2003)
Cheng, Y.: When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika 74, 619–632 (2009)
Chalmers, R.P.: mirt: A multidimensional item response theory package for the R environment. J. Stat. Softw. 48, 1–29 (2012)
Cai, L.: Metropolis-Hastings Robbins-Monro algorithm for confirmatory item factor analysis. J. Educ. Behavi. Stat. 35, 307–335 (2010)
Shute, V., Leighton, J.P., Jang, E.E., Chu, M.-W.: Advances in the science of assessment. Educ. Assess. 21(1), 34–59 (2015)
Leighton, J.P., Gierl, M.J., Hunka, S.M.: The attribute hierarchy method for cognitive assessment: a variation on Tatsuoka’s rule-space approach. J. Educ. Measur. 41, 205–237 (2004)
Rupp, A., Levy, R., Dicerbo, K.E., Sweet, S.J., Crawford, A.V., Calico, T., Benson, M., Fay, D., Kunze, K.L., Mislevy, R.J.: Putting ECD into practice: the interplay of theory and data in evidence models within a digital learning environment. JEDM-J. Educ. Data Min. 4, 49–110 (2012)
Doignon, J.-P., Falmagne, J.-C.: Knowledge Spaces. Springer Science & Business Media, New York (2012)
Lynch, D., Howlin, C.P.: Real world usage of an adaptive testing algorithm to uncover latent knowledge (2014)
Mandin, S., Guin, N.: Basing learner modelling on an ontology of knowledge and skills. In: 2014 IEEE 14th International Conference on Advanced learning technologies (iCALT), pp. 321–323. IEEE (2014)
Kickmeier-Rust, M.D., Albert, D.: Competence-based knowledge space theory. In: Measuring and Visualizing Learning in the Information-Rich Classroom, p. 109 (2015)
Acknowledgements
This work is supported by the Paris-Saclay Institut de la Société Numérique funded by the IDEX Paris-Saclay, ANR-11-IDEX-0003-02.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Vie, JJ., Popineau, F., Bourda, Y., Bruillard, É. (2016). Adaptive Testing Using a General Diagnostic Model. In: Verbert, K., Sharples, M., Klobučar, T. (eds) Adaptive and Adaptable Learning. EC-TEL 2016. Lecture Notes in Computer Science(), vol 9891. Springer, Cham. https://doi.org/10.1007/978-3-319-45153-4_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-45153-4_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-45152-7
Online ISBN: 978-3-319-45153-4
eBook Packages: Computer ScienceComputer Science (R0)