short-paper

Leveraging Large Language Models for Analysis of Student Course Feedback

Authors:

Zixuan Wang,

Paul Denny,

Juho Leinonen,

Andrew Luxton-ReillyAuthors Info & Claims

COMPUTE '23: Proceedings of the 16th Annual ACM India Compute Conference

Pages 76 - 79

https://doi.org/10.1145/3627217.3627221

Published: 19 December 2023 Publication History

Get Access

Abstract

This study investigates the use of large language models, specifically ChatGPT, to analyse the feedback from a Summative Evaluation Tool (SET) used to collect student feedback on the quality of teaching. We find that these models enhance comprehension of SET scores and the impact of context on student evaluations. This work aims to reveal hidden patterns in student evaluation data, demonstrating a positive first step towards automated, detailed analysis of student feedback.

References

[1]

Samuel Cunningham-Nelson, Melinda Laundon, and Abby Cathcart. 2021. Beyond satisfaction scores: visualising student comments for whole-of-course evaluation. Assessment & Evaluation in Higher Education 46, 5 (July 2021), 685–700. https://doi.org/10.1080/02602938.2020.1805409 Publisher: Routledge _eprint: https://doi.org/10.1080/02602938.2020.1805409.

Crossref

Google Scholar

[2]

Robert Godwin-Jones. 2023. Emerging spaces for language learning: AI bots, ambient intelligence, and the metaverse. (Feb. 2023). https://hdl.handle.net/10125/73501 Publisher: University of Hawaii National Foreign Language Resource Center.

Google Scholar

[3]

Leonid Grebennikov and Mahsood Shah. 2013. Student voice: using qualitative feedback from students to enhance their university experience. Teaching in Higher Education 18, 6 (Aug. 2013), 606–618. https://doi.org/10.1080/13562517.2013.774353 Publisher: Routledge _eprint: https://doi.org/10.1080/13562517.2013.774353.

Crossref

Google Scholar

[4]

Henry A. Hornstein. 2017. Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Education 4, 1 (Jan. 2017), 1304016. https://doi.org/10.1080/2331186X.2017.1304016 Publisher: Cogent OA _eprint: https://www.tandfonline.com/doi/pdf/10.1080/2331186X.2017.1304016.

Crossref

Google Scholar

[5]

Neel Kant, Raul Puri, Nikolai Yakovenko, and Bryan Catanzaro. 2018. Practical Text Classification With Large Pre-Trained Language Models. arxiv:1812.01207 [cs.CL]

Google Scholar

[6]

Michael Morgan, Jane Sinclair, Matthew Butler, Neena Thota, Janet Fraser, Gerry Cross, and Jana Jackova. 2018. Understanding International Benchmarks on Student Engagement: Awareness and Research Alignment from a Computer Science Perspective. In Proceedings of the 2017 ITiCSE Conference on Working Group Reports(ITiCSE-WGR ’17). Association for Computing Machinery, New York, NY, USA, 1–24. https://doi.org/10.1145/3174781.3174782

Digital Library

Google Scholar

[7]

Krzysztof Rybinski and Elzbieta Kopciuszewska. 2021. Will artificial intelligence revolutionise the student evaluation of teaching? A big data study of 1.6 million student reviews. Assessment & Evaluation in Higher Education 46, 7 (Oct. 2021), 1127–1139. https://doi.org/10.1080/02602938.2020.1844866 Publisher: Routledge _eprint: https://doi.org/10.1080/02602938.2020.1844866.

Crossref

Google Scholar

[8]

Carly Steyn, Clint Davies, and Adeel Sambo. 2019. Eliciting student feedback for course development: the application of a qualitative course evaluation tool among business research students. Assessment & Evaluation in Higher Education 44, 1 (Jan. 2019), 11–24. https://doi.org/10.1080/02602938.2018.1466266 Publisher: Routledge _eprint: https://doi.org/10.1080/02602938.2018.1466266.

Crossref

Google Scholar

[9]

Wolfgang Stroebe. 2020. Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis. Basic and Applied Social Psychology 42, 4 (July 2020), 276–294. https://doi.org/10.1080/01973533.2020.1756817 Publisher: Routledge _eprint: https://doi.org/10.1080/01973533.2020.1756817.

Crossref

Google Scholar

[10]

Bob Uttl, Carmela A. White, and Daniela Wong Gonzalez. 2017. Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation 54 (Sept. 2017), 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007

Crossref

Google Scholar

[11]

Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces(IUI ’22). Association for Computing Machinery, New York, NY, USA, 841–852. https://doi.org/10.1145/3490099.3511105

Digital Library

Google Scholar

Cited By

View all

Pereira ENsair SPereira LGrant K(2024)Constructive alignment in a graduate-level project management course: an innovative framework using large language modelsInternational Journal of Educational Technology in Higher Education10.1186/s41239-024-00457-221:1Online publication date: 17-Apr-2024
https://doi.org/10.1186/s41239-024-00457-2

Index Terms

Leveraging Large Language Models for Analysis of Student Course Feedback
1. Social and professional topics
  1. Professional topics
    1. Computing education

Recommendations

Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform
WWW '24: Companion Proceedings of the ACM Web Conference 2024

Large language model evaluation plays a pivotal role in the enhancement of its capacity. Previously, numerous methods for evaluating large language models have been proposed in this area. Despite their effectiveness, these existing works mainly focus on ...
Editing Personality For Large Language Models
Natural Language Processing and Chinese Computing
Abstract
This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs). This task seeks to adjust the models’ responses to opinion-related questions on specified topics since an individual’s personality ...
Enhancing Formative Feedback at Scale with the Intelligent Feedback Assistant
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2

Formative feedback spans various domains, from education to businesses and creative endeavours. In educational contexts, feedback enriches students' learning and work quality through reflection. However, providing effective feedback at scale is ...

Comments

Information & Contributors

Information

Published In

COMPUTE '23: Proceedings of the 16th Annual ACM India Compute Conference

December 2023

120 pages

ISBN:9798400708404

DOI:10.1145/3627217

Editors:
Chitra Babu
SSN College of Engineering, Kalavakkam, Tamil Nadu
,
Neeraj Goel
IIT Ropar
,
Amey Karkare
IIT Kanpur

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 December 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

COMPUTE '23

COMPUTE '23: 16th Annual ACM India Compute Conference

December 9 - 11, 2023

Hyderabad, India

Acceptance Rates

Overall Acceptance Rate 114 of 622 submissions, 18%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
100
Total Downloads

Downloads (Last 12 months)85
Downloads (Last 6 weeks)6

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pereira ENsair SPereira LGrant K(2024)Constructive alignment in a graduate-level project management course: an innovative framework using large language modelsInternational Journal of Educational Technology in Higher Education10.1186/s41239-024-00457-221:1Online publication date: 17-Apr-2024
https://doi.org/10.1186/s41239-024-00457-2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Index Terms

Recommendations

Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform

Editing Personality For Large Language Models

Enhancing Formative Feedback at Scale with the Intelligent Feedback Assistant

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Share

Share this Publication link

Share on social media

Affiliations