Article

Incremental natural language processing for HRI

Authors:

Matthias ScheutzAuthors Info & Claims

HRI '07: Proceedings of the ACM/IEEE international conference on Human-robot interaction

Pages 263 - 270

https://doi.org/10.1145/1228716.1228752

Published: 10 March 2007 Publication History

Abstract

Robots that interact with humans face-to-face using natural language need to be responsive to the way humans use language in those situations. We propose a psychologically-inspired natural language processing system for robots which performs incremental semantic interpretation of spoken utterances, integrating tightly with the robot's perceptual and motor systems.

References

[1]

J. F. Allen, B. W. Miller, E. K. Ringger, and T. Sikorski. A robust system for natural spoken dialogue. In A. Joshi and M. Palmer, editors, Proceedings of the Thirty-Fourth Annual Meeting of the Association for Computational Linguistics, pages 62--70, San Francisco, 1996. Morgan Kaufmann Publishers.

Digital Library

[2]

C. G. Chambers, M. K. Tanenhaus, K. M. Eberhard, H. Filip, and G. N. Carlson. Circumscribing referential domains in real-time language comprehension. Journal of Memory and Language, 47(1):30--49, 2002.

[3]

D. DeVault and M. Stone. Domain inference in incremental interpretation. In Proc. ICoS, 2003.

[4]

K. M. Eberhard, M. J. Spivey-Knowlton, J. C. Sedivy, and M. K. Tanenhaus. Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of Psycholinguistic Research, 24:409--436, 1995.

[5]

L. D. Erman, F. Hayes-Roth, V. R. Lesser, and D. R. Reddy. The hearsay-ii speech-understanding system: Integrating knowledge to resolve uncertainty. ACM Computing Surveys, 12(2):213--253, 1980.

Digital Library

[6]

C. Fellbaum, editor. Wordnet: An Electronic Lexical Database. MIT Press, Cambridge, 1998.

[7]

M. V. Ferro and B. A. Dion. Efficient incremental parsing for context-free languages. In Proceedings of the 1994 International Conference on Computer Languages, 1994.

[8]

I. Horswill. Tagged behavior-based architectures: Integrating cognition with embodied activity. IEEE Intelligent Systems, September/October 2001:30--38, 2001.

Digital Library

[9]

Y. Kamide, G. T. M. Altmann, and S. L. Haywood. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language, 49(1):133--156, 2003.

[10]

K. Kipper, H. T. Dang, and M. Palmer. Class-based construction of a verb lexicon. In Proceedings of AAAI 2000, Austin, TX, 2000. AAAI Press.

Digital Library

[11]

O. Lemon, A. Bracy, A. Gruenstein, and S. Peters. Information states in a multi-modal dialogue system for human-robot conversation. In Proceedings of Bi-Dialog, 5th Workshop on Formal Semantics and Pragmatics of Dialogue, pages 57--67, 2001.

[12]

D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91--110, 2004.

Digital Library

[13]

B. Lowerre and R. Reddy. The harpy speech understanding system. In W. A. Lea, editor, Trends in Speech Recognition, pages 340--360. Pretice Hall, 1980.

[14]

D. Mori, S. Matsubara, and Y. Inagaki. Incremental parsing for interactive natural language interface. In IEEE International Conference on Systems, Man, and Cybernetics, 2001.

[15]

A. Newlands, A. H. Anderson, and J. Mullen. Adapting communicative strategies to computer-mediated communication: An analysis of task performance and dialogue structure. Applied Cognitive Psychology, 17(3):325--348, 2003.

[16]

B. O'Conaill, S. Whittaker, and S. Wilbur. Conversations over videoconferences: An evaluation of the spoken aspects of video-mediated communication. Human-Computer Interaction, 8(4):389--428, 1993.

Digital Library

[17]

M. Purver and M. Otsuka. Incremental generation by incremental parsing: Tactical generation in dynamic syntax. In Proceedings of the 6th annual CLUK Research Colloquium, 2003.

[18]

C. P. Rose, A. Roque, D. Bhembe, and K. VanLehn. An efficient incremental architecture for robust interpretation. In Proceedings of Human Language Technology Conference, 2002.

Digital Library

[19]

D. Roy. Learning visually-grounded words and syntax for a scene description task. Computer Speech and Language, 16(3), 2002.

[20]

D. Roy, P. Gorniak, N. Mukherjee, and J. Juster. A trainable spoken language understanding system. In Proceedings of the International Conference of Spoken Language Processing, 2002.

[21]

M. Scheutz. ADE - steps towards a distributed development and runtime environment for complex robotic agent architectures. Applied Artificial Intelligence, 20(4-5), 2006.

[22]

M. Scheutz, K. Eberhard, and V. Andronache. A real-time robotic model of human reference resolution using visual constraints. Connection Science Journal, 16(3):145--167, 2004.

[23]

M. Scheutz, P. Schermerhorn, J. Kramer, and D. Anderson. First steps toward natural human-like hri. Autonomous Robots, page forthcoming, 2007.

Digital Library

[24]

M. Scheutz, P. Schermerhorn, J. Kramer, and C. Middendorff. The utility of affect expression in natural language interactions in joint human-robot tasks. In Proceedings of the 1st ACM International Conference on Human-Robot Interaction, pages 226--233, 2006.

Digital Library

[25]

J. C. Sedivy. Invoking discourse-based contrast sets and resolving syntactic ambiguities. Journal of Memory and Language, 46(2):341--370, 2002.

[26]

L. D. Setlock, S. R. Fussel, and C. Neuwirth. Taking it out of context: collaborating within and across cultures in face-to-face settings and via instant messaging. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, 2004.

Digital Library

[27]

Cmu sphinx speech recognition engines. http://fife.speech.cs.cmu.edu/sphinx/.

[28]

L. Steels and F. Kaplan. Bootstrapping grounded word semantics. In E. J. Briscoe, editor, Linguistic evolution through language acquisition: formal and computational models, pages 53--74. Cambridge University Press, Cambridge, UK, 2002.

[29]

M. K. Tanenhaus, M. J. Spivey-Knowlton, K. M. Eberhard, and J. C. Sedivy. Integration of visual and linguistic information in spoken language comprehension. Science, 268:1632--1634, 1995.

[30]

S. Varges and M. Purver. Robust language analysis and generation for spoken dialogue systems. In Proceedings of the ECAI workshop on Development and Evaluation of Robust Spoken Dialogue Systems for Real Applications, Riva del Garda, Italy, Aug. 2006.

[31]

F. Weng, L. Cavedon, B. Raghunathan, D. Mirkovic, H. Cheng, H. Schmidt, H. Bratt, R. Mishra, S. Peters, L. Zhao, S. Upson, E. Shriberg, and C. Bergmann. A conversational dialogue system for cognitively overloaded users. In Proceedings of the 8th International Conference on Spoken Language Processing, 2004.

[32]

T. Winograd. Understanding Natural Language. Academic Press, 1972.

Digital Library

Cited By

Liu RGuo YJin RZhang X(2024)A Review of Natural-Language-Instructed Robot Execution SystemsAI10.3390/ai50300485:3(948-989)Online publication date: 26-Jun-2024
https://doi.org/10.3390/ai5030048
Williams TMatuszek CMead RDepalma N(2024)Scarecrows in Oz: The Use of Large Language Models in HRIACM Transactions on Human-Robot Interaction10.1145/360626113:1(1-11)Online publication date: 31-Mar-2024
https://dl.acm.org/doi/10.1145/3606261
Li SSiva SMott TWilliams TZhang HDantam N(2023)Failure Explanation in Privacy-Sensitive Contexts: An Integrated Systems Approach2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309501(2328-2335)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309501
Show More Cited By

Index Terms

Incremental natural language processing for HRI
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Towards Closed Feedback Loops in HRI: Integrating InproTK and PaMini
MMRWHRI '14: Proceedings of the 2014 Workshop on Multimodal, Multi-Party, Real-World Human-Robot Interaction

In this paper, we present a first step towards incremental processing for modeling asynchronous human-robot interactions, to allow closed feedback loops in HRI. We achieve this by combining the incremental natural language processing framework InproTK ...
Incremental Unit Networks for Distributed, Symbolic Multimodal Processing and Representation
Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Health, Operations Management, and Design
Abstract
Incremental dialogue processing has been an important topic in spoken dialogue systems research, but the broader research community that makes use of language interaction (e.g., chatbots, conversational AI, spoken interaction with robots) have not ...
Towards incremental speech generation in conversational systems

This paper presents a model of incremental speech generation in practical conversational systems. The model allows a conversational system to incrementally interpret spoken input, while simultaneously planning, realising and self-monitoring the system ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '07: Proceedings of the ACM/IEEE international conference on Human-robot interaction

March 2007

392 pages

ISBN:9781595936172

DOI:10.1145/1228716

General Chairs:
Cynthia Breazeal
Massachusetts Institute of Technology, USA
,
Alan C. Schultz
Naval Research Laboratory, USA
,
Program Chairs:
Terry Fong
NASA Ames Research Center, USA
,
Sara Kiesler
Carnegie Mellon University, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

HRI07

Sponsor:

HRI07: International Conference on Human Robot Interaction

March 10 - 12, 2007

Virginia, Arlington, USA

Acceptance Rates

HRI '07 Paper Acceptance Rate 22 of 101 submissions, 22%;

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

51
Total Citations
View Citations
756
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu RGuo YJin RZhang X(2024)A Review of Natural-Language-Instructed Robot Execution SystemsAI10.3390/ai50300485:3(948-989)Online publication date: 26-Jun-2024
https://doi.org/10.3390/ai5030048
Williams TMatuszek CMead RDepalma N(2024)Scarecrows in Oz: The Use of Large Language Models in HRIACM Transactions on Human-Robot Interaction10.1145/360626113:1(1-11)Online publication date: 31-Mar-2024
https://dl.acm.org/doi/10.1145/3606261
Li SSiva SMott TWilliams TZhang HDantam N(2023)Failure Explanation in Privacy-Sensitive Contexts: An Integrated Systems Approach2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309501(2328-2335)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309501
Jackson RWilliams T(2022)Enabling Morally Sensitive Robotic Clarification RequestsACM Transactions on Human-Robot Interaction10.1145/350379511:2(1-18)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3503795
Scheutz MThielstrom RAbrams M(2022)Transparency through Explanations and Justifications in Human-Robot Task-Based CommunicationsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.209108638:18-20(1739-1752)Online publication date: 27-Jul-2022
https://doi.org/10.1080/10447318.2022.2091086
Blake Jackson RLi SBalajee Banisetty SSiva SZhang HDantam NWilliams T(2021)An Integrated Approach to Context-Sensitive Moral Cognition in Robot Cognitive Architectures2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS51168.2021.9636434(1911-1918)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1109/IROS51168.2021.9636434
Hinaut XTwiefel J(2020)Teach Your Robot Your Language! Trainable Neural Parser for Modeling Human Sentence Processing: Examples for 15 LanguagesIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2019.295700612:2(179-188)Online publication date: Jun-2020
https://doi.org/10.1109/TCDS.2019.2957006
Biswas MRomeo MCangelosi AJones R(2019)Are older people any different from younger people in the way they want to interact with robots? Scenario based surveyJournal on Multimodal User Interfaces10.1007/s12193-019-00306-x14:1(61-72)Online publication date: 24-Jul-2019
https://doi.org/10.1007/s12193-019-00306-x
Williams TYazdani FSuresh PScheutz MBeetz M(2019)Dempster-Shafer theoretic resolution of referential ambiguityAutonomous Robots10.1007/s10514-018-9795-543:2(389-414)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s10514-018-9795-5
Moetesum MSiddiqi I(2018)Socially Believable RobotsHuman-Robot Interaction - Theory and Application10.5772/intechopen.71375Online publication date: 4-Jul-2018
https://doi.org/10.5772/intechopen.71375
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents