Abstract
We propose and evaluate a question-answering system that uses decomposed prompting to classify and answer student questions on a course discussion board. Our system uses a large language model (LLM) to classify questions into one of four types: conceptual, homework, logistics, and not answerable. This enables us to employ a different strategy for answering questions that fall under different types. Using a variant of GPT-3, we achieve 81% classification accuracy. We discuss our system’s performance on answering conceptual questions from a machine learning course and various failure modes.
Supported by Vector Institute, NSERC, Fujitsu, Amazon Research Award, and the CIFAR AI Chairs Program.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Drori, I., et al.: A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proc. Natl. Acad. Sci. 119(32), e2123433119 (2022)
Feng, D., Shaw, E., Kim, J., Hovy, E.: An intelligent discussion-bot for answering student queries in threaded discussions. In: Proceedings of the 11th international conference on Intelligent user interfaces, pp. 171–177 (2006)
Goel, A.K., Polepeddi, L.: Jill watson: A virtual teaching assistant for online education. In: Learning Engineering For Online Education, pp. 120–143, Routledge (2018)
Khot, T., et al.: Decomposed prompting: A modular approach for solving complex tasks (2022). https://doi.org/10.48550/ARXIV.2210.02406
Ouyang, L., et al.: Training language models to follow instructions with human feedback. arXiv:2203.02155 (2022)
Sarsa, S., Denny, P., Hellas, A., Leinonen, J.: Automatic generation of programming exercises and code explanations using large language models. In: Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 1, pp. 27–43 (2022)
Wang, Z., Valdez, J., Basu Mallick, D., Baraniuk, R.G.: Towards human-like educational question generation with large language models. In: Artificial Intelligence in Education: 23rd International Conference, AIED 2022, Durham, UK, July 27–31, 2022, Proceedings, Part I, pp. 153–166, Springer (2022). https://doi.org/10.1007/978-3-031-11644-5_13
Zylich, B., Viola, A., Toggerson, B., Al-Hariri, L., Lan, A.: Exploring automated question answering methods for teaching assistance. In: Bittencourt, I.I., Cukurova, M., Muldner, K., Luckin, R., Millán, E. (eds.) AIED 2020. LNCS (LNAI), vol. 12163, pp. 610–622. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52237-7_49
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jaipersaud, B., Zhang, P., Ba, J., Petersen, A., Zhang, L., Zhang, M.R. (2023). Decomposed Prompting to Answer Questions on a Course Discussion Board. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-36336-8_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36335-1
Online ISBN: 978-3-031-36336-8
eBook Packages: Computer ScienceComputer Science (R0)