Abstract
We propose a novel classification method of recognized second language learners utterances into three classes of acceptability for dialogue-based computer assisted language learning (CALL) systems. Our method uses a linear classifier trained with three types of bilingual evaluation understudy (BLEU) scores. The three BLEU scores are calculated respectively, referring to three subsets of a learner corpus divided according to the quality of sentences. Our method classifies learner utterances into three classes (correct, acceptable with some modifications and out-of-the-scope of assumed erroneous sentences), since it is suitable for providing effective feedback. Experimental results showed that our proposed classification method could distinguish utterance acceptability with 75.8 % accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Seneff, S., Wang, C.: Web-based dialogue and translation games for spoken language learning. In: SLaTE, pp. 9–16 (2007)
Wik, P., Hjalmarsson, A.: Embodied conversational agents in computer assisted language learning. Speech Commun. 51(10), 1024–1037 (2009)
Kyusong, L., Kweon, S.O., Sungjin, L., Hyungjong, N., Lee, G.G.: Postech immersive english study (POMY): dialog-based language learning game. IEICE TRANS. Inf. Syst. 97(7), 1830–1841 (2014)
Raux, A., Eskenazi, M.: Using task-oriented spoken dialogue systems for language learning: potential, practical applications and challenges. In: InSTIL/ICALL Symposium 2004 (2004)
Anzai, T., Ito, A.: Recognition of utterances with grammatical mistakes based on optimization of language model towards interactive CALL systems. In: Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Asia-Pacific, pp. 1–4. IEEE (2012)
Nagai, Y., Senzai, T., Yamamoto, S., Nishida, M.: Sentence classification with grammatical errors and those out of scope of grammar assumption for dialogue-based CALL systems. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2012. LNCS, vol. 7499, pp. 616–623. Springer, Heidelberg (2012)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Lin, C.Y., Och, F.J.: Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p. 605. Association for Computational Linguistics (2004)
Minematsu, N., Tomiyama, Y., Yoshimoto, K., Shimizu, K., Nakagawa, S., Dantsuji, M., Makino, S.: Development of English speech database read by Japanese to support CALL research. In: Proceedings ICA, vol. 1, pp. 557–560 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Kuwa, R., Wang, X., Kato, T., Yamamoto, S. (2016). Classification of Utterance Acceptability Based on BLEU Scores for Dialogue-Based CALL Systems. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds) Text, Speech, and Dialogue. TSD 2016. Lecture Notes in Computer Science(), vol 9924. Springer, Cham. https://doi.org/10.1007/978-3-319-45510-5_58
Download citation
DOI: https://doi.org/10.1007/978-3-319-45510-5_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-45509-9
Online ISBN: 978-3-319-45510-5
eBook Packages: Computer ScienceComputer Science (R0)