Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Human Tutorial Instruction in the Raw

Published: 25 March 2015 Publication History

Abstract

Humans learn procedures from one another through a variety of methods, such as observing someone do the task, practicing by themselves, reading manuals or textbooks, or getting instruction from a teacher. Some of these methods generate examples that require the learner to generalize appropriately. When procedures are complex, however, it becomes unmanageable to induce the procedures from examples alone. An alternative and very common method for teaching procedures is tutorial instruction, where a teacher describes in general terms what actions to perform and possibly includes explanations of the rationale for the actions. This article provides an overview of the challenges in using human tutorial instruction for teaching procedures to computers. First, procedures can be very complex and can involve many different types of interrelated information, including (1) situating the instruction in the context of relevant objects and their properties, (2) describing the steps involved, (3) specifying the organization of the procedure in terms of relationships among steps and substeps, and (4) conveying control structures. Second, human tutorial instruction is naturally plagued with omissions, oversights, unintentional inconsistencies, errors, and simply poor design. The article presents a survey of work from the literature that highlights the nature of these challenges and illustrates them with numerous examples of instruction in many domains. Major research challenges in this area are highlighted, including the difficulty of the learning task when procedures are complex, the need to overcome omissions and errors in the instruction, the design of a natural user interface to specify procedures, the management of the interaction of a human with a learning system, and the combination of tutorial instruction with other teaching modalities.

References

[1]
S. Adams. 2008. The future of end user programming? In Proceedings of the International Conference on Software Engineering.
[2]
J. F. Allen, N. Chambers, G. Ferguson, L. Galescu, H. Jung, M. Swift, and W. Taysom. 2007. PLOW: A collaborative task learning agent. In Proceedings of the National Conference on Artificial Intelligence (AAAI’07).
[3]
R. Alterman, R. Zito-Wolf, and T. Carpenter. 1991. Interaction, comprehension, and instruction usage. Journal of the Learning Sciences 1, 4, 361--398.
[4]
C. Anderson. 2008. The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion.
[5]
I. Androutsopoulos, G. D. Ritchie, and P. Thanisch. 1995. Natural language interfaces to databases—an introduction. Natural Language Engineering 1, 1, 28--81.
[6]
C. L. Baker, C. J. Fillmore, and J. B. Lowe. 1998. The Berkeley framenet project. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics, Vol. 1. 86--90.
[7]
J. Blythe and Y. Gil. 2004. Incremental formalization of document annotations through ontology-based paraphrasing. Proceedings of the 13th International World Wide Web Conference (WWW’04).
[8]
B. W. Boehm, C. Abts, W. Brown, S. Chulani, B. K. Clark, E. Horowitz, R. Madachy, D. J. Reifer, and B. Steece. 2000. Software Cost Estimation with COCOMO II. Prentice Hall.
[9]
M. A. Britt and A. A. Larson. 2003. Constructing representations of arguments. Journal of Memory and Language 48, 4, 794--810.
[10]
F. Brooks. 1987. No silver bullet: Essence and accidents of software engineering. IEEE Computer 20, 4, 10--19.
[11]
P. Brown and J. Gould. 1987. An experimental study of people creating spreadsheets. ACM Transactions on Office Information Systems 5, 3, 258--272.
[12]
G. Bugmann, S. Lauria, T. Kyriacou, E. Klein, J. Bos, and K. Coventry. 2001. Using verbal instructions for route learning: Instruction analysis. In Proceedings of the Conference Towards Autonomous Robots (TIMR’01). 1--6.
[13]
N. J. Butko and J. Movellan. 2007. Learning to learn. In Proceedings of the 6th International Conference on Development and Learning (ICDL’07).
[14]
V. Castelli, L. D. Bergman, T. Lau, and D. Oblinger. 2010. Sheepdog, parallel collaborative programming-by-demonstration. Knowledge-Based Systems 23, 2, 94--109.
[15]
J. Chen and D. S. Weld. 2008. Recovering from errors during programming by demonstration. In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI’08).
[16]
M. T. H. Chi, S. A. Siler, H. Jeong, T. Yamauchi, and R. G. Hausmann. 2001. Learning from human tutoring. Cognitive Science 25, 4, 471--533.
[17]
M. Chi and K. VanLehn. 2008. Eliminating the gap between the high and low students through meta-cognitive strategy instruction. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems.
[18]
B. Chandrasekaran, J. Glasgow, and N. H. Narayanan (Eds.). 1995. Diagrammatic Reasoning. AAAI Press.
[19]
P. Clark, P. Harrison, T. Jenkins, J. Thompson, and R. Wojcik. 2005. Acquiring and using world knowledge using a restricted subset of English. In Proceedings of the 18th International Conference of the Florida Artificial Intelligence Research Society (FLAIRS’05).
[20]
P. Clark, J. Thompson, K. Barker, B. Porter, V. Chaud-Hri, A. Rodriguez, J. Thomere, S. Mishra, Y. Gil, P. Hayes, and T. Reichherzer. 2001. Knowledge entry as the graphical assembly of components. In Proceedings of the International Conference on Knowledge Capture (K-CAP’01).
[21]
R. E. Clark, D. F. Feldon, J. J. G. Van Merrienboer, K. Yates, and S. Early. 2007. Cognitive task analysis. In Handbook of Research on Educational Communications and Technology (3rd ed.). Lawrence Erlbaum, 577--593.
[22]
B. Di Eugenio. 1998. An action representation formalism to interpret natural language instructions. Computational Intelligence 14, 1.
[23]
B. Di Eugenio and B. L. Webber. 1996. Pragmatic overloading in natural language instructions. International Journal of Expert Systems 9, 1, 53--84.
[24]
P. Dixon. 1982. Plans and directions for complex tasks. Journal of Verbal Learning and Verbal Behavior 21.
[25]
P. Dixon. 1987. The processing of organizational and component step information in written directions. Journal of Memory and Language 26, 24--35.
[26]
P. Dixon, J. Faries, and G. Gabrys. 1998. The role of explicit action statements in understanding and using written directions. Journal of Memory and Language 27, 6, 649--667.
[27]
J. Donin, R. J. Bracewell, C. Fredericksen, and M. Dillinger. 1992. Students’ strategies for writing instructions. Written Communication 9, 2, 209--236.
[28]
B. Eylon and F. Reif. 1984. Effects of knowledge organization on task performance. Cognition and Instruction 1, 1, 5--44.
[29]
J. Feldman. 2000. Minimization of Boolean complexity in human concept learning. Nature 407, 630--633.
[30]
C. J. Fillmore. 1968. The case for case. In Universals in Linguistic Theory, E. Bach and R. T. Harms (Eds.). Holt, Rinehart, and Winston, 1--25.
[31]
C. Fritz and Y. Gil. 2011. A formal framework for combining natural instruction and demonstration for end-user programming. In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI’11).
[32]
N. E. Fuchs and R. Schwitter. 1996. Attempto controlled English (ACE). In Proceedings of the 1st International Workshop on Controlled Language Applications (CLAW’96).
[33]
N. E. Fuchs, K. Kaljurand, and G. Schneider. 2006. Attempto controlled English meets the challenges of knowledge representation, reasoning, interoperability and user interfaces. In Proceedings of the International Conference of the Florida Artificial Intelligence Research Society (FLAIRS’06).
[34]
G. W. Furnas, T. K. Landauer, L. M. Gomez, and S. T. Dumais. 1987. The vocabulary problem in human-system communication. Communications of the ACM 30, 11, 964--971.
[35]
K. M. Galotti and W. F. Ganong. 1985. What non-programmers know about programming: Natural language procedure specification. International Journal of Man-Machine Studies 22, 1, 1--10.
[36]
Y. Gil and E. Melz. 1996. Explicit representations of problem-solving strategies to support knowledge acquisition. Proceedings of the 13th National Conference on Artificial Intelligence (AAAI’96).
[37]
Y. Gil, V. Ratnakar, and C. Fritz. 2011. TellMe: Learning procedures from tutorial instruction. In Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI’11).
[38]
Y. Gil, V. Ratnakar, T. Chklovski, P. Groth, and D. Vrandecic. 2012. Capturing common knowledge about tasks: Intelligent assistance for to do lists. ACM Transactions on Interactive Intelligent Systems 2, 3, Article No. 15.
[39]
K. Gold and B. Scassellati. 2007. A robot that uses existing vocabulary to infer non-visual word meanings from observation. In Proceedings of the 22nd Annual Meeting of the Association for the Advancement of Artificial Intelligence (AAAI’07).
[40]
K. Gold, M. Doniec, and B. Scassellati. 2007. Learning grounded semantics with word trees: Prepositions and pronouns. In Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL’07).
[41]
T. R. G. Green and M. Petre. 1996. Usability analysis of visual programming environments: A ‘cognitive dimensions’ framework. Journal of Visual Languages and Computing 7, 2, 131--174.
[42]
S. Greene, S. Devlin, P. Cannata, and L. Gomez. 1990. No IFs, ANDs, or ORs: A study of database querying. International Journal of Man-Machine Studies 32, 3, 303--326.
[43]
H. P. Grice. 1975. Logic and conversation. In Syntax and Semantics III: Speech Acts, P. Cole and J. Morgan (Eds.). Academic Press, New York, NY.
[44]
J. S. Herberg, M. M. Saylor, P. Ratanaswasd, D. T. Levin, and D. M. Wilkes. 2008. Audience-contingent variation in action demonstrations for humans and computers. Cognitive Science 32, 6, 1003--1020.
[45]
J. M. Hoc. 1989. Do we really have conditional statements in our brains? In Studying the Novice Programmer, E. Soloway and J. C. Spohrer (Eds.). Lawrence Erlbaum Associates.
[46]
S. Huffman and J. Laird. 1995. Flexibly instructable agents. Journal of Artificial Intelligence Research 3, 1, 271--324.
[47]
J. Huo and W. Cowan. 2008. Comprehending Boolean queries. In Proceedings of the 5th ACM Symposium on Applied Perception in Graphics and Visualization.
[48]
B. E. John and D. E. Kieras. 1996. Using GOMS for user interface design and evaluation: Which technique? ACM Transactions on Computer-Human Interaction 3, 4.
[49]
A. Kay. 1984. Computer software. Scientific American 251, 3.
[50]
C. Kelleher and R. Pausch. 2005. Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers. ACM Computing Surveys 37, 2, 83--137.
[51]
D. E. Kieras and S. Bovair. 1984. The role of a mental model in learning to operate a device. Cognitive Science 8, 3.
[52]
E. Kim and B. Scassellati. 2007. Learning to refine behavior using prosodic feedback. In Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL’07).
[53]
E. S. Kim, D. Leyzberg, K. M. Tsui, and B. Scassellati. 2009. How people talk when teaching a robot. In Proceedings of the 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI’09).
[54]
J. Kim and Y. Gil. 2000. User studies of an interdependency-based interface for acquiring problem-solving knowledge. In Proceedings of the International Conference on Intelligent User Interfaces (IUI’00).
[55]
P. A. Kirschner, J. Sweller, and R. E. Clark. 2006. Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist 41, 2.
[56]
A J. Ko, B. A. Myers, and D. H. Chau. 2006. A linguistic analysis of how people describe software problems. In Proceedings of the Visual Languages and Human-Centric Computing Conference (VL/HCC’06).
[57]
A. J. Ko, R. Abraham, L. Beckwith, A. Blackwell, M. M. Burnett, M. Erwig, C. Scaffidi, J. Lawrence, H. Lieberman, B. A. Myers, M. B. Rosson, G. Rothermel, M. Shaw, and S. Wiedenbeck. 2011. The state of the art in end-user software engineering. ACM Computing Surveys 43, 3, Article No. 21.
[58]
K. Koedinger and J. Anderson. 1992. Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science 14, 4, 511--550.
[59]
L. Kosseim and G. Lapalme. 1995. Choosing rhetorical relations in instructional texts: The case of effects and guidances. In Proceedings of The Fifth European Workshop on Natural Language Generation.
[60]
L. Kosseim and G. Lapalme. 2000. Choosing rhetorical structures to plan instructional texts. Computational Intelligence 16, 3, 408--445.
[61]
J. Larkin and H. A. Simon. 1987. Why a diagram is (sometimes) worth 10,000 words. Cognitive Science 11, 1, 65--100.
[62]
T. Lau, C. Drews, and J. Nichols. 2009. Interpreting written how-to instructions. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI’09).
[63]
S. Lauria, G. Bugmann, T. Kyriacou, and E. Klein. 2002. Mobile robot programming using natural language. Robotics and Autonomous Systems 38, 3, 171--181.
[64]
M. D. Lee and M. J. Dry. 2006. Decision making and confidence given uncertain advice. Cognitive Science 30, 6, 1081--1095.
[65]
J. LeFevre and P. Dixon. 1986. Do written instructions need examples? Cognition and Instruction 3, 1, 1--30.
[66]
I. Li, J. Nichols, T. A. Lau, C. Drews, and A. Cypher. 2010. Here's what I did: Sharing and reusing Web activity with ActionShot. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI’10).
[67]
H. Lieberman, F. Paterno, and, V. Wulf (Eds.). 2005. End-User Development. Kluwer/Springer.
[68]
K. V. Linden. 1994. Generating precondition expressions in instructional text. In Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics.
[69]
K. V. Linden and B. Di Eugenio. 1996. A corpus study of negative imperatives in natural language instructions. In Proceedings of the 16th International Conference on Computational Linguistics (COLING’96).
[70]
K. V. Linden and J. H. Martin. 1995. Expressing rhetorical relations in instructional text: A case study of the purpose relation. Computational Linguistics 21, 1, 29--57.
[71]
D. E. Mahling and W. B. Croft. 1988. Relating human knowledge of tasks to the requirements of plan libraries. International Journal of Man-Machine Studies 31, 1, 61--97.
[72]
D. E. Mahling and W. B. Croft. 1993. Acquisition and support of goal-based tasks. Knowledge Acquisition 5, 1, 37--77.
[73]
C. W. Mann and S. A. Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text 8, 3.
[74]
L. A. Miller. 1974. Programming by non-programmers. International Journal of Man-Machine Studies 6, 237--260.
[75]
L. A. Miller. 1981. Natural language programming: Styles, strategies, and contrasts. IBM Systems Journal 20, 2, 184--215.
[76]
M. L. Miller. 1979. A structured planning and debugging environment for elementary programming. International Journal of Man-Machine Studies 11, 1, 79--95.
[77]
B. A. Myers, J. F. Pane, and A. Ko. 2004. Natural programming languages and environments. Communications of the ACM 47, 9, 47--52.
[78]
B. A. Nardi. 1995. A Small Matter of Programming: Perspectives on End User Computing. MIT Press, Cambridge, MA.
[79]
E. Nyberg and T. Mitamura. 1996. Controlled language and knowledge-based machine translation: Principles and practice. In Proceedings of the 1st International Workshop on Controlled Language Applications (CLAW’96).
[80]
L. A. Onorato and R. W. Schvaneveldt. 1987. Programmer-nonprogrammer differences in specifying procedures to people and computers. Journal of Systems and Software 7, 4, 357--369.
[81]
J. Pane and B. Myers. 2000a. The influence of the psychology of programming on a language design. In Proceedings of the 12th Annual Meeting of the Psychology of Programmers Interest Group.
[82]
J. F. Pane and B. A. Myers. 2000b. Tabular and textual methods for selecting objects from a group. In Proceedings of IEEE International Symposium on Visual Languages.
[83]
C. M. Pietras and B. G. Coury. 1994. The development of cognitive models of planning for use in the design of project management systems. International Journal of Human-Computer Studies 40, 1, 5--30.
[84]
M. Resnick, J. Maloney, A. Monroy-Hernández, N. Rusk, E. Eastmond, K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, and Y. Kafai. 2009. Scratch: Programming for all. Communications of the ACM 52, 11, 60--67.
[85]
C. Scaffidi, M. Shaw, and B. Myers. 2005. Estimating the numbers of end user programmers. In Proceedings of the 2005 Symposium on Visual Languages and Human-Centric Computing (VL/HCC’05).
[86]
D. L. Schwartz and J. D. Bransford. 1998. A time for telling. Cognition and Instruction 16, 4, 475--522.
[87]
J. D. Slotta and M. T. H. Chi. 2006. Helping students understand challenging topics in science through ontology training. Cognition and Instruction 24, 2, 261--289.
[88]
E. Smith and L. Goodman. 1984. Understanding written instructions: The role of an explanatory schema. Cognition and Instruction 1, 4, 359--396.
[89]
E. Soloway, J. Bonar, and K. Ehrlich. 1983. Cognitive strategies and looping constructs: An empirical study. Communications of the ACM 26, 11, 853--860.
[90]
M. Steehouder, J. Karreman, and N. Ummelen. 2000. Making sense of step-by-step procedures. In Proceedings of IEEE Professional Communication Society International Professional Communication Conference and the 18th Annual ACM International Conference on Computer Documentation: Technology and Teamwork.
[91]
W. R. Swartout, C. Paris, and J. Moore. 1991. Explanations in knowledge systems: Design for explainable expert systems. IEEE Expert 6, 3, 47--49.
[92]
A. L. Thomaz and C. Breazeal. 2008a. Teachable robots: Understanding human teaching behavior to build more effective robot learners. Artificial Intelligence 172, 6--7, 716--737.
[93]
A. L. Thomaz and C. Breazeal. 2008b. Experiments in socially guided exploration: Lessons learned in building robots that learn with and without human teachers. Connection Science: Social Learning in Embodied Agents 20, 2--3, 91--110.
[94]
A. L. Thomaz and M. Cakmak. 2009. Learning about objects with human teachers. In Proceedings of the International Conference on Human Robot Interaction (HRI’09).
[95]
K. VanLehn, A. C. Graesser, G. T. Jackson, P. Jordan, A. Olney, and C. P. Rose. 2007. When are tutorial dialogues more effective than reading? Cognitive Science 31, 1, 3--62.
[96]
J. J. G. Van Merriënboer. 1997. Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training. Educational Technology Pubns.
[97]
J. J. G. Van Merrienboer, P. A. Kirschner, and L. Kester. 2003. Taking the load of a learner's mind: Instructional design for complex learning. Educational Psychologist 38, 1, 5--13.
[98]
T. Walker, G. Kunapuli, N. Larsen, D. Page, and J. Shavlik. 2011. Integrating knowledge capture and supervised learning through a human-computer interface. In Proceedings of the 6th International Conference on Knowledge Capture (KCAP’11).
[99]
B. L. Webber, N. Badler, B. Di Eugenio, C. W. Geib, L. Levison, and M. Moore. 1995. Instructions, intentions and expectations. Artificial Intelligence 73, 1--2, 253--269.
[100]
W. Weiland and B. Shneiderman. 1993. A graphical query interface based on aggregation/generalization hierarchies. Information Systems 18, 4, 215--232.
[101]
T. Winograd and F. Flores. 1986. Understanding Computers and Cognition: A New Foundation for Design. Ablex Publishing, Norwood, NJ.
[102]
P. Wright and A. J. Hull. 1990. How people give verbal instructions. Applied Cognitive Psychology 4, 3, 153--174.
[103]
D. Young and B. Shneiderman. 1993. A graphical filter/flow representation of Boolean queries: A prototype implementation and evaluation. Journal of American Society for Information Science 44, 6, 327--339.
[104]
M. Young. 1999. Using Grice's maxim of quantity to select the content of plan descriptions. Artificial Intelligence 115, 2, 215--256.

Cited By

View all
  • (2021)Unpacking Human Teachers’ Intentions for Natural Interactive Task Learning2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)10.1109/RO-MAN50785.2021.9515448(1173-1180)Online publication date: 8-Aug-2021
  • (2017)Enriching how-to guides with actionable phrases and linked dataWeb Intelligence10.3233/WEB-17036415:3(189-203)Online publication date: 11-Aug-2017
  • (2017)Towards Automating Data NarrativesProceedings of the 22nd International Conference on Intelligent User Interfaces10.1145/3025171.3025193(565-576)Online publication date: 7-Mar-2017
  • Show More Cited By

Index Terms

  1. Human Tutorial Instruction in the Raw

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Interactive Intelligent Systems
    ACM Transactions on Interactive Intelligent Systems  Volume 5, Issue 1
    March 2015
    164 pages
    ISSN:2160-6455
    EISSN:2160-6463
    DOI:10.1145/2744352
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 March 2015
    Accepted: 01 June 2013
    Revised: 01 February 2013
    Received: 01 September 2011
    Published in TIIS Volume 5, Issue 1

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Procedure learning
    2. end-user programming
    3. instruction
    4. intelligent user interfaces
    5. interactive learning
    6. natural language interpretation

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)9
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 03 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2021)Unpacking Human Teachers’ Intentions for Natural Interactive Task Learning2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)10.1109/RO-MAN50785.2021.9515448(1173-1180)Online publication date: 8-Aug-2021
    • (2017)Enriching how-to guides with actionable phrases and linked dataWeb Intelligence10.3233/WEB-17036415:3(189-203)Online publication date: 11-Aug-2017
    • (2017)Towards Automating Data NarrativesProceedings of the 22nd International Conference on Intelligent User Interfaces10.1145/3025171.3025193(565-576)Online publication date: 7-Mar-2017
    • (2016)Enriching How-to Guides by Linking Actionable PhrasesProceedings of the 25th International Conference Companion on World Wide Web10.1145/2872518.2890585(939-944)Online publication date: 11-Apr-2016

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media