Abstract
AutoTutor is a learning environment that tutors students by holding a conversation in natural language. AutoTutor has been developed for Newtonian qualitative physics and computer literacy. Its design was inspired by explanation-based constructivist theories of learning, intelligent tutoring systems that adaptively respond to student knowledge, and empirical research on dialogue patterns in tutorial discourse. AutoTutor presents challenging problems (formulated as questions) from a curriculum script and then engages in mixed initiative dialogue that guides the student in building an answer. It provides the student with positive, neutral, or negative feedback on the student’s typed responses, pumps the student for more information, prompts the student to fill in missing words, gives hints, fills in missing information with assertions, identifies and corrects erroneous ideas, answers the student’s questions, and summarizes answers. AutoTutor has produced learning gains of approximately .70 sigma for deep levels of comprehension.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Aleven, V., &Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science,26,147–179.
Allen, J. (1995).Natural language understanding. Redwood City, CA: Benjamin/Cummings.
Anderson, J. R., Corbett, A. T., Koedinger, K. R., &Pelletier, R. (1995). Cognitive tutors: Lessons learned.Journal of the Learning Sciences,4,167–207.
Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents.Journal of Educational Psychology,94,416–427.
Bloom, B. S. (Ed.) (1956).Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: Longmans, Green.
Burgess, C., Livesay, K., &Lund, K. (1998). Explorations in context space: Words, sentences, and discourse.Discourse Processes,25, 211–257.
Cassell, J., &Thorisson, K. (1999). The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents.Applied Artificial Intelligence,13,519–538.
Chi, M. T. H., de Leeuw, N., Chiu, M., &LaVancher, C. (1994). Eliciting self-explanations improves understanding.Cognitive Science,18,439–477.
Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., &Hausmann, R. G. (2001). Learning from human tutoring.Cognitive Science,25,471–533.
Cohen, P. A., Kulik, J. A., &Kulik, C. C. (1982). Educational outcomes of tutoring: A meta-analysis of findings.American Educational Research Journal,19,237–248.
Colby, K. M., Weber, S., &Hilf, F. D. (1971). Artificial paranoia.Artificial Intelligence,2, 1–25.
Collins, A., Brown, J. S., &Newman, S. E. (1989). Cognitive apprenticeship: Teaching the craft of reading, writing, and mathematics. In L. B. Resnick (Ed.),Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Erlbaum.
Collins, A., Warnock, E. H., &Passafiume, J. J. (1975). Analysis and synthesis of tutorial dialogues. In G. H. Bower (Ed.),The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 49–87). New York: Academic Press.
DARPA (1995).Proceedings of the Sixth Message Understanding Conference (MUC-6). San Francisco: Morgan Kaufman.
Foltz, P. W., Gilliam, S., &Kendall, S. (2000). Supporting content-based feedback in on-line writing evaluation with LSA.Interactive Learning Environments,8,111–128.
Fox, B. (1993).The human tutorial dialogue project. Hillsdale, NJ: Erlbaum.
Graesser, A. C., Gernsbacher, M. A., &Goldman, S. (Eds.) (2003).Handbook of discourse processes. Mahwah, NJ: Erlbaum.
Graesser, A. C., Hu, X., & McNamara, D. S. (in press). Computerized learning environments that incorporate research in discourse psychology, cognitive science, and computational linguistics. In A. F. Healy (Ed.),Experimental cognitive psychology and its applications: Festschrift in honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer. Washington, DC: American Psychological Association.
Graesser, A. C., Jackson, G. T., Mathews, E. C., Mitchell, H. H., Olney, A., Ventura, M., &Chipman, P. (2003). Why/AutoTutor: A test of learning gains from a physics tutor with natural language dialog. In R. Alterman & D. Hirsh (Eds.),Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society (pp. 1–6). Mahwah, NJ: Erlbaum.
Graesser, A. C., Moreno, K., Marineau, J., Adcock, A., Olney, A., Person, N., &the Tutoring Research Group (2003). AutoTutor improves deep learning of computer literacy: Is it the dialogue or the talking head? In U. Hoppe, F. Verdejo, & J. Kay (Eds.),Proceedings of artificial intelligence in education (pp. 47–54). Amsterdam: IOS Press.
Graesser, A. C., &Olde, B. A. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down.Journal of Educational Psychology,95, 524–536.
Graesser, A. C., &Person, N. K. (1994). Question asking during tutoring.American Educational Research Journal,31104–137.
Graesser, A. C., Person, N. K., Harter, D., &the Tutoring Research Group (2001). Teaching tactics and dialogue in AutoTutor.International Journal of Artificial Intelligence in Education,12, 257–279.
Graesser, A. C., Person, N. K., &Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-on-one tutoring.Applied Cognitive Psychology,9, 359–387.
Graesser, A. C., VanLehn, K., Rosé, C. P., Jordan, P. W., &Harter, D. (2001). Intelligent tutoring systems with conversational dialogue.AI Magazine,22(4), 39–52.
Graesser, A. C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R., &the Tutoring Research Group (1999). AutoTutor: A simulation of a human tutor.Journal of Cognitive Systems Research,1,35–51.
Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N. K., &the Tutoring Research Group (2000). Using latent semantic analysis to evaluate the contributions of students in AutoTutor.Interactive Learning Environments,8,129–148.
Gratch, J., Rickel, J., André, E., Cassell, J., Petajan, E., &Badler, N. (2002). Creating interactive virtual humans: Some assembly required.IEEE Intelligent Systems,17,54–63.
Harabagiu, S. M., Maiorano, S. J., &Pasca, M. A. (2002). Open-domain question answering techniques.Natural Language Engineering,1, 1–38.
Heffernan, N. T., &Koedinger, K. R. (1998). A developmental model for algebra symbolization: The results of a difficulty factor assessment. In M. A. Gernsbacher & S. J. Derry (Eds.),Proceedings of the Twentieth Annual Conference of the Cognitive Science Society (pp. 484–489). Mahwah, NJ: Erlbaum.
Hu, X., Cai, Z., Graesser, A. C., Louwerse, M. M., Penumatsa, P., Olney, A., &the Tutoring Research Group (2003). An improved LSA algorithm to evaluate student contributions in tutoring dialogue. In G. Gottlob & T. Walsh (Eds.),Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (pp. 1489–1491). San Francisco: Morgan Kaufmann.
Hume, G. D., Michael, J. A., Rovick, A., &Evens, M. W. (1996). Hinting as a tactic in one-on-one tutoring.Journal of the Learning Sciences,5, 23–47.
Johnson, W. L., Rickel, J. W., &Lester, J. C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments.International Journal of Artificial Intelligence in Education,11, 47–78.
Jurafsky, D., &Martin, J. H. (2000).Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Upper Saddle River, NJ: Prentice Hall.
Kintsch, E., Steinhart, D., Stahl, G., &the LSA Research Group (2000). Developing summarization skills through the use of LSA-based feedback.Interactive Learning Environments,8,87–109.
Kintsch, W. (1998).Comprehension: A paradigm for cognition. New York: Cambridge University Press.
Landauer, T. K., &Dumais, S. T. (1997). An answer to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review,104,211–240.
Landauer, T. K., Foltz, P. W., &Laham, D. (1998). An introduction to latent semantic analysis.Discourse Processes,25, 259–284.
Lehnert, W. G., &Ringle, M. H. (Eds.) (1982). Strategies for natural language processing. Hillsdale, NJ: Erlbaum.
Lenat, D. B. (1995). CYC: A large-scale investment in knowledge infrastructure.Communications of the ACM,38,33–38.
Louwerse, M. M., Graesser, A. C., Olney, A., &the Tutoring Research Group (2002). Good computational manners: Mixed-initiative dialogue in conversational agents. In C. Miller (Ed.),Etiquette for human—computer work: Papers from the 2002 fall symposium, Technical Report FS-02-02 (pp. 71–76). North Falmouth, MA: AAAI Press.
Louwerse, M. M., &Mitchell, H. H. (2003). Towards a taxonomy of a set of discourse markers in dialog: A theoretical and computational linguistic account.Discourse Processes,35, 199–239.
Louwerse, M. M., &van Peer, W. (Eds.) (2002).Thematics: Interdisciplinary studies. Philadelphia: John Benjamins.
Manning, C. D., &Schütze, H. (1999).Foundations of statistical natural language processing. Cambridge, MA: MIT Press.
Massaro, D. W., &Cohen, M. M. (1995). Perceiving talking faces.Current Directions in Psychological Science,4, 104–109.
Moore, J. D. (1995).Participating in explanatory dialogues. Cambridge, MA: MIT Press.
Moore, J. D., &Wiemer-Hastings, P. (2003). Discourse in computational linguistics and artificial intelligence. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.),Handbook of discourse processes (pp. 439–486). Mahwah, NJ: Erlbaum.
Moreno, K. N., Klettke, B., Nibbaragandla, K., Graesser, A. C., &the Tutoring Research Group (2002). Perceived characteristics and pedagogical efficacy of animated conversational agents. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 963–971). Berlin: Springer-Verlag.
Moreno, R., Mayer, R. E., Spires, H. A., &Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents?Cognition & Instruction,19,177–213.
Norman, D. A., &Rumelhart, D. E. (1975).Explorations in cognition. San Francisco: Freeman.
Olde, B. A., Franceschetti, D. R., Karnavat, A., Graesser, A. C., &the Tutoring Research Group (2002). The right stuff: Do you need to sanitize your corpus when using latent semantic analysis? In W. D. Gray & C. D. Schunn (Eds.),Proceedings of the Twenty-Fourth Annual Meeting of the Cognitive Science Society (pp. 708–713). Mahwah, NJ: Erlbaum.
Olney, A., Louwerse, M. M., Mathews, E. C., Marineau, J., Mitchell, H. H., &Graesser, A. C. (2003). Utterance classification in AutoTutor. In J. Burstein & C. Leacock (Eds.),Building educational applications using natural language processing: Proceedings of the Human Language Technology, North American chapter of the Association for Computational Linguistics Conference 2003 Workshop (pp. 1–8). Philadelphia: Association for Computational Linguistics.
Palincsar, A. S., &Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities.Cognition & Instruction,1,117–175.
Person, N. K., Graesser, A. C., Bautista, L., Mathews, E. C., &the Tutoring Research Group (2001). Evaluating student learning gains in two versions of AutoTutor. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.),Artificial intelligence in education: AI-ED in the wired and wireless future (pp. 286–293). Amsterdam: IOS Press.
Person, N. K., Graesser, A. C., Kreuz, R. J., Pomeroy, V., &the Tutoring Research Group (2001). Simulating human tutor dialogue moves in AutoTutor.International Journal of Artificial Intelligence in Education,12, 23–39.
Person, N. K., Graesser, A. C., &the Tutoring Research Group (2002). Human or computer? AutoTutor in a bystander Turing test. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 821–830). Berlin: Springer-Verlag.
Picard, R. W. (1991).Affective computing. Cambridge, MA: MIT Press.
Rich, C., &Sidner, C. L. (1998). COLLAGEN: A collaborative manager for software interface agents.User Modeling & User-Adapted Interaction,8,315–350.
Rickel, J., Lesh, N., Rich, C., Sidner, C. L., &Gertner, A. S. (2002). Collaborative discourse theory as a foundation for tutorial dialogue. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 542–551). Berlin: Springer-Verlag.
Schank, R. C., &Riesbeck, C. K. (Eds.) (1982).Inside computer understanding: Five Programs Plus Miniatures. Hillsdale, NJ: Erlbaum.
Sekine, S., &Grishman, R. (1995). A corpus-based probabilistic grammar with only two non-terminals. In H. Bunt (Ed.),Fourth international workshop on parsing technology (pp. 216–223). Prague: Association for Computational Linguistics.
Shah, F., Evens, M. W., Michael, J., &Rovick, A. (2002). Classifying student initiatives and tutor responses in human keyboard-to-keyboard tutoring sessions.Discourse Processes,33, 23–52.
Sleeman, D., &Brown, J. S. (Eds.) (1982).Intelligent tutoring systems. New York: Academic Press.
Song, K., Hu, X., Olney, A., Graesser, A. C., & the Tutoring Research Group (in press). A framework of synthesizing tutoring conversation capability with Web based distance education courseware.Computers & Education.
Susarla, S., Adcock, A., Van Eck, R., Moreno, K., Graesser, A. C., &the Tutoring Research Group (2003). Development and evaluation of a lesson authoring tool for AutoTutor. In V. Aleven, U. Hoppe, J. Kay, R. Mizoguchi, H. Pain, F. Verdejo, & K. Yacef (Eds.),AIED2003 supplemental proceedings (pp. 378–387). Sydney: University of Sydney School of Information Technologies.
VanLehn, K., & Graesser, A. C. (2002).Why2 report: Evaluation of Why/Atlas, Why/AutoTutor, and accomplished human tutors on learning gains for qualitative physics problems and explanations. Unpublished report prepared by the University of Pittsburgh CIRCLE group and the University of Memphis Tutoring Research Group.
VanLehn, K., Jones, R. M., &Chi, M. T. H. (1992). A model of the self-explanation effect.Journal of the Learning Sciences,2, 1–60.
VanLehn, K., Jordan, P., Rosé, C. P., Bhembe, D., Bottner, M., Gaydos, A., Makatchev, M., Pappuswamy, U., Ringenberg, M., Roque, A., Siler, S., &Srivastava, R. (2002). The architecture of Why2-Atlas: A coach for qualitative physics essay writing. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 158–167). Berlin: Springer-Verlag.
VanLehn, K., Lynch, C., Taylor, L., Weinstein, A., Shelby, R., Schulze, K., Treacy, D., &Wintersgill, M. (2002). Minimally invasive tutoring of complex physics problem solving. In S. A. Cerri, G. Gouarderes, & F. Paraguacu (Eds.),Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (pp. 367–376). Berlin: Springer-Verlag.
Voorhees, E. M. (2001). The TREC question answering track.Natural Language Engineering,7,361–378.
Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine.Communications of the ACM,9, 36–45.
Whittaker, S. (2003). Theories and methods in mediated communication. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.),Handbook of discourse processes (pp. 243–286). Mahwah, NJ: Erlbaum.
Winograd, T. (1972).Understanding natural language. New York: Academic Press.
Woods, W. A. (1977). Lunar rocks in natural English: Explorations in natural language question answering. In A. Zampoli (Ed.),Linguistic structures processing (pp. 201–222). New York: Elsevier.
Author information
Authors and Affiliations
Corresponding author
Additional information
The Tutoring Research Group is an interdisciplinary research team comprised of approximately 35 researchers with backgrounds in psychology, computer science, physics, and education (for more information, visithttp://www.autotutor.org). The research on AutoTutor was supported by National Science Foundation (NSF) Grants SBR 9720314, REC 0106965, REC 0126265, and ITR 0325428 and by Grant N00014-00-1-0600 of the Department of Defense Multidisciplinary University Research Initiative administered by the Office of Naval Research (ONR). Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the Department of Defense, the ONR, or the NSF. Kurt VanLehn, Carolyn Rosé, Pam Jordan, and others at the University of Pittsburgh collaborated with us in preparing AutoTutor materials on conceptual physics.
Rights and permissions
About this article
Cite this article
Graesser, A.C., Lu, S., Jackson, G.T. et al. AutoTutor: A tutor with dialogue in natural language. Behavior Research Methods, Instruments, & Computers 36, 180–192 (2004). https://doi.org/10.3758/BF03195563
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03195563