Hacia una mejora en la práctica docente utilizando el Análisis de Sentimiento en la Evaluación de Estudiantes
Barra lateral del artículo
Contenido principal del artículo
La evaluación del estudiante sobre la enseñanza (SET) es una forma ad hoc de evaluar la efectividad docente en instituciones de educación superior. En este documento, presentamos un enfoque para analizar los sentimientos expresados en los comentarios de SET utilizando un modelo de lenguaje grande (LLM). Al emplear técnicas de procesamiento de lenguaje natural, extraemos y analizamos los sentimientos expresados por los estudiantes al finalizar el curso, con el objetivo de proporcionar a educadores y administradores ideas valiosas sobre la calidad de la enseñanza y elementos a mejorar de la práctica docente. Nuestro estudio demuestra la efectividad de los LLM en el análisis de sentimientos de los comentarios, resaltando su potencial para mejorar el proceso de evaluación. Nuestros experimentos con un conjunto de datos etiquetados de forma colaborativa demuestran un 93% de precisión en la clasificación de los mensajes. Discutimos las implicaciones de nuestros hallazgos para las instituciones educativas y proponemos futuras direcciones para la investigación en este ámbito.
Omer K, Jacobs S, Bettger B, Dawson J, Graether S, Murrant C, et al. Evaluating and Improving the Formative Use of Student Evaluations of Teaching. Can J Scholarsh Teach Learn. 2023;14(1):n1. DOI: https://doi.org/10.5206/cjsotlrcacea.2023.1.10960
Mancenido Z. Impact evaluations of teacher preparation practices: Challenges and opportunities for more rigorous research. Rev Educ Res. 2023;00346543231174413. DOI: https://doi.org/10.3102/1883286
Cunningham S, Laundon M, Cathcart A, Bashar MA, Nayak R. First, do no harm: automated detection of abusive comments in student evaluation of teaching surveys. Assess Eval High Educ. 2023;48(3):377–89. DOI: https://doi.org/10.1080/02602938.2022.2081668
Gravestock P, Gregor-Greenleaf E. Student course evaluations: Research, models and trends. Higher Education Quality Council of Ontario Toronto; 2008.
Uttl B. Lessons learned from research on student evaluation of teaching in higher education. Stud Feedback Teach Sch Using Stud Percept Dev Teach Teach. 2021;237–56. DOI: https://doi.org/10.1007/978-3-030-75150-0_15
Gaoa G, Pangb M, Pengc J, Lud Y. A Hierarchical Probe Evaluation Method for Teaching Effect of University Engineering Courses Based on the Keypoints of Knowledge. In: 3rd International Conference on Education, Language and Art (ICELA 2023). Atlantis Press; 2024. p. 376–84. DOI: https://doi.org/10.2991/978-2-38476-214-9_45
Alshammari E. Student evaluation of teaching. Is it valid? J Adv Pharm Educ Res Apr-Jun. 2020;10(2):97.
Röhl S, Bijlsma H, Rollett W. The process model of student feedback on teaching (SFT): A theoretical framework and introductory remarks. Stud Feedback Teach Sch Using Stud Percept Dev Teach Teach. 2021;1–11. DOI: https://doi.org/10.1007/978-3-030-75150-0_1
Jensen E, Dale M, Donnelly PJ, Stone C, Kelly S, Godley A, et al. Toward automated feedback on teacher discourse to enhance teacher learning. In: Proceedings of the 2020 chi conference on human factors in computing systems. 2020. p. 1–13. DOI: https://doi.org/10.1145/3313831.3376418
Mandouit L, Hattie J. Revisiting “The Power of Feedback” from the perspective of the learner. Learn Instr. 2023;84:101718. DOI: https://doi.org/10.1016/j.learninstruc.2022.101718
Lim LA, Dawson S, Gašević D, Joksimović S, Pardo A, Fudge A, et al. Students’ perceptions of, and emotional responses to, personalised learning analytics-based feedback: an exploratory study of four courses. Assess Eval High Educ. 2021;46(3):339–59. DOI: https://doi.org/10.1080/02602938.2020.1782831
Chen S, Nieminen JH. Towards an ecological understanding of student emotions in feedback: a scoping review. Assess Eval High Educ. 2024;1–18. DOI: https://doi.org/10.1080/02602938.2024.2318641
Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) [Internet]. Minneapolis, Minnesota: Association for Computational Linguistics; 2019. p. 4171–86. Available from: https://aclanthology.org/N19-1423
Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language Models are Unsupervised Multitask Learners. 2019;
Altrabsheh N, Cocea M, Fallahkhair S. Learning sentiment from students’ feedback for real-time interventions in classrooms. In: International conference on adaptive and intelligent systems. Springer; 2014. p. 40–9. DOI: https://doi.org/10.1007/978-3-319-11298-5_5
Shaikh S, Daudpota SM, Yayilgan SY, Sindhu S. Exploring the potential of large-language models (LLMs) for student feedback sentiment analysis. In: 2023 International Conference on Frontiers of Information Technology (FIT). IEEE; 2023. p. 214–9. DOI: https://doi.org/10.1109/FIT60620.2023.00047
Häkkinen J, Ramadan Z. A Study on the Perception of Feedback with Varying Sentiment Generated Using a Large Language Model. 2023.
Mouronte-López ML, Ceres JS, Columbrans AM. Analysing the sentiments about the education system trough Twitter. Educ Inf Technol. 2023;28(9):10965–94. DOI: https://doi.org/10.1007/s10639-022-11493-8
Shen L, Wang M, Shen R. Affective e-learning: Using “emotional” data to improve learning in pervasive learning environment. J Educ Technol Soc. 2009;12(2):176–89.
Zhou J, Ye J min. Sentiment analysis in education research: a review of journal publications. Interact Learn Environ. 2023;31(3):1252–64. DOI: https://doi.org/10.1080/10494820.2020.1826985
Cutroni L, Paladino A. Peer-ing in: A systematic review and framework of peer review of teaching in higher education. Teach Teach Educ. 2023;133:104302. DOI: https://doi.org/10.1016/j.tate.2023.104302
Zhu JJ, Chang YC, Ku CH, Li SY, Chen CJ. Online critical review classification in response strategy and service provider rating: Algorithms from heuristic processing, sentiment analysis to deep learning. J Bus Res. 2021;129:860–77. DOI: https://doi.org/10.1016/j.jbusres.2020.11.007
Cox A. Exploring the impact of Artificial Intelligence and robots on higher education through literature-based design fictions. Int J Educ Technol High Educ. 2021;18(1):1–19. DOI: https://doi.org/10.1186/s41239-020-00237-8
Tian F, Zheng Q, Zhao R, Chen T, Jia X. Can e-Learner’s emotion be recognized from interactive Chinese texts? In: 2009 13th International Conference on Computer Supported Cooperative Work in Design. 2009. p. 546–51.
Ortigosa A, Martín JM, Carro RM. Sentiment analysis in Facebook and its application to e-learning. Comput Hum Behav. 2014;31:527–41. DOI: https://doi.org/10.1016/j.chb.2013.05.024
Poulos A, Mahony MJ. Effectiveness of feedback: The students’ perspective. Assess Eval High Educ. 2008;33(2):143–54. DOI: https://doi.org/10.1080/02602930601127869
Gan Z, He J, Zhang LJ, Schumacker R. Examining the relationships between feedback practices and learning motivation. Meas Interdiscip Res Perspect. 2023;21(1):38–50. DOI: https://doi.org/10.1080/15366367.2022.2061236
Kanwar A, Sanjeeva M. Student satisfaction survey: A key for quality improvement in the higher education institution. J Innov Entrep. 2022;11(1):27. DOI: https://doi.org/10.1186/s13731-022-00196-6
Giang NTP, Dien TT, Khoa TTM. Sentiment analysis for university students’ feedback. In: Advances in Information and Communication: Proceedings of the 2020 Future of Information and Communication Conference (FICC), Volume 2. Springer; 2020. p. 55–66. DOI: https://doi.org/10.1007/978-3-030-39442-4_5
Dalipi F, Zdravkova K, Ahlgren F. Sentiment analysis of students’ feedback in MOOCs: A systematic literature review. Front Artif Intell. 2021;4:728708. DOI: https://doi.org/10.3389/frai.2021.728708
Neumann M, Linzmayer R. Capturing student feedback and emotions in large computing courses: A sentiment analysis approach. In: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 2021. p. 541–7. DOI: https://doi.org/10.1145/3408877.3432403
Ren P, Yang L, Luo F. Automatic scoring of student feedback for teaching evaluation based on aspect-level sentiment analysis. Educ Inf Technol. 2023;28(1):797–814. DOI: https://doi.org/10.1007/s10639-022-11151-z
Katragadda S, Ravi V, Kumar P, Lakshmi GJ. Performance analysis on student feedback using machine learning algorithms. In: 2020 6th international conference on advanced computing and communication systems (ICACCS). IEEE; 2020. p. 1161–3. DOI: https://doi.org/10.1109/ICACCS48705.2020.9074334
Mabunda JGK, Jadhav A, Ajoodha R. Sentiment analysis of student textual feedback to improve teaching. In: Interdisciplinary Research in Technology and Management. CRC Press; 2021. p. 643–51. DOI: https://doi.org/10.1201/9781003202240-100
Reddy SS, Gadiraju M, Maheswara Rao V. Analyzing student reviews on teacher performance using long short-term memory. In: Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2021. Springer; 2022. p. 539–53. DOI: https://doi.org/10.1007/978-981-16-7167-8_39
Agostini D, Picasso F. Large Language Models for Sustainable Assessment and Feedback in Higher Education: Towards a Pedagogical and Technological Framework. In: Proceedings of the First International Workshop on High-Performance Artificial Intelligence Systems in Education Co-Located with 22nd International Conference of the Italian Association for Artificial Intelligence (AIxIA 2023). 2023.
Parker MJ, Anderson C, Stone C, Oh Y. A large language model approach to educational survey feedback analysis. ArXiv Prepr ArXiv230917447. 2023;
Freyd M. A graphic rating scale for teachers. J Educ Res. 1923;8(5):433–9. DOI: https://doi.org/10.1080/00220671.1923.10879421
Boring A, Ottoboni K. Student evaluations of teaching (mostly) do not measure teaching effectiveness. Sci Res. 2016; DOI: https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1
Hoel A, Dahl TI. Why bother? Student motivation to participate in student evaluations of teaching. Assess Eval High Educ. 2019;44(3):361–78. DOI: https://doi.org/10.1080/02602938.2018.1511969
Chen Y. Does students’ evaluation of teaching improve teaching quality? Improvement versus the reversal effect. Assess Eval High Educ. 2023;48(8):1195–207. DOI: https://doi.org/10.1080/02602938.2023.2177252
Newman H, Joyner D. Sentiment Analysis of Student Evaluations of Teaching. In: Penstein Rosé C, Martínez-Maldonado R, Hoppe HU, Luckin R, Mavrikis M, Porayska-Pomsta K, et al., editors. Artificial Intelligence in Education. Cham: Springer International Publishing; 2018. p. 246–50.
Dake DK, Gyimah E. Using sentiment analysis to evaluate qualitative students’ responses. Educ Inf Technol. 2023;28(4):4629–47. DOI: https://doi.org/10.1007/s10639-022-11349-1
Falcon S, Leon J. How do teachers engaging messages affect students? A sentiment analysis. Educ Technol Res Dev. 2023;71(4):1503–23. DOI: https://doi.org/10.1007/s11423-023-10230-3
Bird S, Loper E. NLTK: The Natural Language Toolkit. In: Proceedings of the ACL Interactive Poster and Demonstration Sessions [Internet]. Barcelona, Spain: Association for Computational Linguistics; 2004 [cited 2021 Jul 3]. p. 214–7. Available from: https://aclanthology.org/P04-3031
Honnibal M, Montani I. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. Appear. 2018;
Pérez JM, Giudici JC, Luque F. pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks. 2021.
Pérez JM, Furman DA, Alonso Alemany L, Luque FM. RoBERTuito: a pre-trained language model for social media text in Spanish. In: Proceedings of the Thirteenth Language Resources and Evaluation Conference [Internet]. Marseille, France: European Language Resources Association; 2022. p. 7235–43. Available from: https://aclanthology.org/2022.lrec-1.785
García-Vega M, Díaz-Galiano M, García-Cumbreras M, Del Arco F, Montejo-Ráez A, Jiménez-Zafra S, et al. Overview of TASS 2020: Introducing emotion detection. In: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2020) Co-Located with 36th Conference of the Spanish Society for Natural Language Processing (SEPLN 2020), Málaga, Spain. 2020. p. 163–70.
Hutto C, Gilbert E. VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. Proc Int AAAI Conf Web Soc Media. 2014 May;8(1):216–25. DOI: https://doi.org/10.1609/icwsm.v8i1.14550
Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. ArXiv Prepr ArXiv191001108. 2019;
Adoma AF, Henry NM, Chen W. Comparative analyses of bert, roberta, distilbert, and xlnet for text-based emotion recognition. In: 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). IEEE; 2020. p. 117–21. DOI: https://doi.org/10.1109/ICCWAMTIP51612.2020.9317379
Aceptado 2024-06-17
Publicado 2024-06-20
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-CompartirIgual 4.0.
Los autores que publican en esta revista están de acuerdo con los siguientes términos:
Los autores ceden los derechos patrimoniales a la revista y a la Universidad del Valle sobre los manuscritos aceptados, pero podrán hacer los reusos que consideren pertinentes por motivos profesionales, educativos, académicos o científicos, de acuerdo con los términos de la licencia que otorga la revista a todos sus artículos.
Los artículos serán publicados bajo la licencia Creative Commons 4.0 BY-NC-SA (de atribución, no comercial, sin obras derivadas).