Verification and Validation of Adaptive Instructional Systems: A Text Mining Review
Author: 
Bruno EmondAuthors Info & Claims
Bruno EmondAuthors Info & ClaimsPages 25 - 43
Abstract
The current paper aims at qualifying the distribution of academic papers related to verification and validation of Adaptive Instructional Systems (AIS) and Adaptive software (ASOFT). For the purpose of the literature review, the theme of software verification and validation is divided into three sub-themes: 1) software verification, 2) empirical validation, and 3) model and simulation validation. In order to maintain the paper broad objectives, the approach will apply text mining techniques to analyze the literature. The corpus contained 33 546 documents extracted from Scopus (January 2024) in 5 document sets that were analyzed using TF-IDF vectors to measure cosine similarity between documents. The similarity distributions as well as the statistical tests indicate a difference in attention given to verification and validation issues between publications in AIS and ASOFT. The more important difference between the two sets of documents is the role given to software verification. The stronger emphasis on empirical, and model and simulation validation in the AIS literature points to the high dependency of considering human factors in the success of adaptive instructional systems. In regard to software verification in AIS, the few publications addressing this issue in comparison to publications looking at tutoring software engineering techniques indicate a possible research and practical gap to explore. The brief overview of the extended number of publications in software verification of ASOFT did not provide an immediate and clear set of methods and techniques that could be applied in the context of AIS. A further analysis could look in more depth in the ASOFT literature or at the AI development engineering literature.
References
[1]
Acikgul Firat E and Firat S Web 3.0 in learning environments: a systematic review Turk. Online J. Dist. Educ. 2021 22 1 148-169
[2]
Aguilar R, Muñoz V, Noda M, Bruno A, and Moreno L Verification and validation of an intelligent tutorial system Expert Syst. Appl. 2008 35 3 677-685
[3]
Barker, S., Douglas, P.: An intelligent tutoring system for program semantics, vol. 1, pp. 482–487 (2005).
[4]
Bell B and Sottilare R Sottilare RA and Schwarz J Adaptation vectors for instructional agents Adaptive Instructional Systems 2019 Cham Springer 3-14
[5]
Bodily R and Verbert K Review of research on student-facing learning analytics dashboards and educational recommender systems IEEE Trans. Learn. Technol. 2017 10 4 405-418
[6]
Cully A and Demiris Y Online knowledge level tracking with data-driven student models and collaborative filtering IEEE Trans. Knowl. Data Eng. 2020 32 10 2000-2013
[7]
Das Dôres, S., Alves, L., Ruiz, D., Barros, R.: A meta-learning framework for algorithm recommendation in software fault prediction, vol. 04-08-April-2016, pp. 1486–1491 (2016).
[8]
Oliveira de Almeida H, Dias da Silva L, Perkusich A, and de Barros Costa E Choren R, Garcia A, Lucena C, and Romanovsky A A formal approach for the modelling and verification of multiagent plans based on model checking and petri nets Software Engineering for Multi-Agent Systems III 2005 Heidelberg Springer 162-179
[9]
Emond B Sottilare RA and Schwarz J Formal methods in human-computer interaction and adaptive instructional systems Adaptive Instructional Systems. Design and Evaluation 2021 Cham Springer 183-198
[10]
Fazlollahi, A.: AI in surgical curriculum design and unintended outcomes for technical competencies in simulation training. JAMA network open 6(9), e2334658 (2023).
[11]
Ferrando, A., Dennis, L., Cardoso, R., Fisher, M., Ancona, D., Mascardi, V.: Toward a holistic approach to verification and validation of autonomous cognitive systems. ACM Trans. Softw. Eng. Methodol. 30(4) (2021).
[12]
Fontaine, G., Cossette, S., Maheu-Cadotte, M.A., Mailhot, T., Deschênes, M.F., Mathieu-Dupuis, G.: Effectiveness of adaptive e-learning environments on knowledge, competence, and behavior in health professionals and students: protocol for a systematic review and meta-analysis. JMIR Res. Protocols 6(7) (2017).
[13]
Forejt V, Kwiatkowska M, Parker D, Qu H, and Ujma M Qadeer S and Tasiran S Incremental runtime verification of probabilistic systems Runtime Verification 2013 Heidelberg Springer 314-319
[14]
Gaspar-Figueiredo, D., Abrahao, S., Insfran, E., Vanderdonckt, J.: Measuring user experience of adaptive user interfaces using EEG: a replication study, pp. 52–61 (2023).
[15]
Haendler T, Neumann G, and Smirnov F Lane HC, Zvacek S, and Uhomoibhi J RefacTutor: an interactive tutoring system for software refactoring Computer Supported Education 2020 Cham Springer 236-261
[16]
Heitmeyer C Petrenko A, Simão A, and Maldonado JC A model-based approach to testing software for critical behavior and properties Testing Software and Systems 2010 Heidelberg Springer 15
[17]
Hoq, M., Chilla, S., Ranjbar, M., Brusilovsky, P., Akram, B.: SANN: programming code representation using attention neural network with optimized subtree extraction, pp. 783–792 (2023).
[18]
Hou, J.: Formal semantic model for agent-based software system (2010).
[19]
Hussain J et al. Model-based adaptive user interface based on context and user experience evaluation J. Multimodal User Interfaces 2018 12 1 1-16
[20]
Huynh, N.T., Segarra, M.T., Beugnard, A.: A development process based on variability modeling for building adaptive software architectures, pp. 1715–1718 (2016).
[21]
Imtiaz, S., Sultana, K., Varde, A.: Mining learner-friendly security patterns from huge published histories of software applications for an intelligent tutoring system in secure coding, pp. 4869–4876 (2021).
[22]
Kang, H.A., Sales, A., Whittaker, T.: Flow with an intelligent tutor: a latent variable modeling approach to tracking flow during artificial tutoring. Behav. Res. Methods (2023).
[23]
Koedinger K and Aleven V An interview reflection on “intelligent tutoring goes to school in the big city” Int. J. Artif. Intell. Educ. 2016 26 1 13-24
[24]
Kooken J, Zaini R, and Arroyo I Simulating the dynamics of self-regulation, emotion, grit, and student performance in cyber-learning environments Metacogn. Learn. 2021 16 2 367-405
[25]
Lee E, Kim YG, Seo YD, Seol K, and Baik DK RINGA: design and verification of finite state machine for self-adaptive software at runtime Inf. Softw. Technol. 2018 93 200-222
[26]
Leite, W., et al.: Heterogeneity of treatment effects of a video recommendation system for algebra, pp. 12–23 (2022).
[27]
Luburić, N., et al.: Clean code tutoring: makings of a foundation, vol. 1, pp. 137–148 (2022).
[28]
Maggioni, S., Lünenburger, L., Riener, R., Curt, A., Bolliger, M., Melendez-Calderon, A.: Assessing walking ability using a robotic gait trainer: opportunities and limitations of assist-as-needed control in spinal cord injury. J. NeuroEng. Rehabil. 20(1) (2023).
[29]
Menor, J.: Design, development and effectiveness of an intelligent tutoring system using neural network, vol. 2602 (2023).
[30]
O’Mara-Eves A, Thomas J, McNaught J, Miwa M, and Ananiadou S Using text mining for study identification in systematic reviews: a systematic review of current approaches Systems Control Found. Appl. 2015 4 1 5
[31]
Rajaraman, A., Ullman, J.D.: Data Mining, p. 1–17. Cambridge University Press (2011)
[32]
Rebai, R., Maalej, M., Mahfoudhi, A., Abid, M.: Bayesian user modeling: evaluation metrics of an adaptive user interface, vol. 10341 (2017).
[33]
Rim R, Mohamed Amin M, Adel M, and Mohamed A Madureira AM, Abraham A, Gamboa D, and Novais P Evaluation method for an adaptive web interface: GOMS model Intelligent Systems Design and Applications 2017 Cham Springer 116-124
[34]
Sancenon, V., et al.: A new web-based personalized learning system improves student learning outcomes. Int. J. Virtual Pers. Learn. Environ. 12(1) (2022).
[35]
Sghaier, O., Sahraoui, H.: A multi-step learning approach to assist code review, pp. 450–460 (2023).
[36]
Shneiderman B Bridging the gap between ethics and practice: guidelines for reliable, safe, and trustworthy human-centered AI systems ACM Trans. Interact. Intell. Syst. 2020 10 4 1-31
[37]
Sifakis J and Harel D Trustworthy autonomous system development ACM Trans. Embed. Comput. Syst. 2023 22 3 1-24
[38]
Sottilare R, Knowles A, and Goodell J Sottilare RA and Schwarz J Representing functional relationships of adaptive instructional systems in a conceptual model Adaptive Instructional Systems 2020 Cham Springer 176-186
[39]
Sottilare RA Sottilare RA and Schwarz J Establishing an effective adaptive instructional systems community of practice Adaptive Instructional Systems 2023 Cham Springer 76-93
[40]
Sottilare, R.A., Barr, A., Graesser, A., Hu, X., Robson, R.: Exploring the opportunities and benefits of standards for adaptive instructional systems (AISs). In: Sottilare, R.A., Barr, A., Graesser, A., Hu, X., Robson, R. (eds.) Proceedings of the 1st Adaptive Instructional System (AIS) Standards Workshop. U.S. Army Research Laboratory (2018)
[41]
Soui M, Diab S, Ouni A, Essayeh A, and Abed M Shakhovska N An ontology-based approach for user interface adaptation Advances in Intelligent Systems and Computing 2017 Cham Springer 199-215
[42]
Sundaram G and Berleant D Yang XS, Sherratt RS, Dey N, and Joshi A Automating systematic literature reviews with natural language processing and text mining: a systematic literature review Proceedings of Eighth International Congress on Information and Communication Technology 2023 Singapore Springer 73-92
[43]
Thakur K and Kumar V Application of text mining techniques on scholarly research articles: methods and tools New Rev. Acad. Librariansh. 2022 28 3 279-302
[44]
Thapliyal M, Ahuja N, Shankar A, Cheng X, and Kumar M A differentiated learning environment in domain model for learning disabled learners J. Comput. High. Educ. 2022 34 1 60-82
[45]
Wang H et al. Examining the applications of intelligent tutoring systems in real educational contexts: a systematic literature review from the social experiment perspective Educ. Inf. Technol. 2023 28 7 9113-9148
[46]
Wang, J., Yang, C., Fu, E., Ngai, G., Leong, H.: Is your mouse attracted by your eyes: non-intrusive stress detection in off-the-shelf desktop environments. Eng. Appl. Artif. Intell. 123 (2023).
[47]
Wang X, Tang X, Dong Z, and Zhen L Research on rapid development platform of plc control system High Technol. Lett. 2021 27 2 210-217
[48]
Younis, E., Zaki, S., Kanjo, E., Houssein, E.: Evaluating ensemble learning methods for multi-modal emotion recognition using sensor data fusion. Sensors 22(15) (2022).
Recommendations
MetaVal: Witness Validation via Verification
Computer Aided VerificationAbstractWitness validation is an important technique to increase trust in verification results, by making descriptions of error paths (violation witnesses) and important parts of the correctness proof (correctness witnesses) available in an exchangeable ...
Efficient Verification of Sequential and Concurrent C Programs
There has been considerable progress in the domain of software verification over the last few years. This advancement has been driven, to a large extent, by the emergence of powerful yet automated abstraction techniques such as predicate abstraction. ...
Software Verification and Validation: An Overview
An explanation is given of software verification and validation (V&V) and how it fits in the development life cycle. How to apply V&V is also discussed. Evaluations of its effectiveness are summarized.
Comments
Information & Contributors
Information
Published In
Jun 2024
367 pages
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 29 June 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 17 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in