Abstract
In natural language queries to an intelligent multimodal system, ambiguities related to referring expressions - source ambiguities - can occur between items in the visual display and objects in the domain being represented. A multimodal interface has to be able to resolve these ambiguities in order to provide satisfactory communication with a user. In this paper, we briefly introduce source ambiguities, and present the formalisation of a constraint satisfaction approach to interpreting singular referring expressions with source ambiguities. In our approach, source ambiguities are resolved simultaneously with other referent ambiguities, allowing flexible access to various sorts of knowledge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
.André, E., Rist, T.: Referring to World Objects with Text and Pictures. In Proceedings of GOLING’94 Kyoto Japan (1994) 530–534.
Ben Amara, H., Peroche, B., Chappel, H., Wilson, M.: Graphical Interaction in a Multimodal Interface. In Proceedings of Esprit Conferences, Kluwer Academic Publisher, Dordrecht, Netherlands (1991) 303–321.
Binot, J., Debille, L., Sedlock, D., Vandecapelle, B., Chappel, H., Wilson, M.: Multimodal integration in MMI2: Anaphora Resolution and Mode Selection. In Luczak, H., Cakir, A., Cakir, G. (eds.): Work With Display Units-WWDU’92 Berlin, Germany (1992).
Haddock, N.: Incremental Semantics and Interactive Syntactic Processing. PhD thesis, University of Edinburgh (1988).
He, D.: References to Graphical Objects in Interactive Multimodal Queries. PhD thesis, University of Edinburgh (2000).
He, D., Ritchie, G., Lee, J.: Referring to Displays in Multimodal Interfaces. In Referring Phenomena in a Multimedia Context and Their Computational Treatment, A workshop of ACL/EACL’97 Madrid Spain (1997) 79–82.
He, D., Ritchie, G., Lee, J.: Disambiguation between Visual Display and Represented Domain in Multimodal Interfaces. In Combining AI and Graphics for the Interface of the Future, A workshop of ECAI’98 Brighton UK (1998) 17–28.
Mackworth, A.: Consistency in Networks of Relations. Artificial Intelligence 8 (1977) 99–118.
McKeown, K., Feiner, S., Robin, J., Seligmann, D., Tanenblatt, M.: Generating Cross-References for Multimedia Explanation. In Proceedings of AAAI’92 San Jose USA (1992) 9–16.
Mellish, C: Computer Interpretation of Natural Language Descriptions. Ellis Horwood series in Artificial Intelligence. Ellis Horwood, 1985.
Neal, J., Shapiro, S.: Intelligent Multimedia Interface Technology. In Sullivan, J., Tyler, S. (eds.) Intelligent User Interfaces ACM Press New York (1991) 11–44.
Rich, E., Knight, K. Artificial Intelligence. McGraw-Hill, New York, 2 Ed (1991).
Walker, M.: Centering, Anaphora Resolution, and Discourse Structure. In Walker, M., Joshi, A., Prince, E. (eds.): Centering Theory in Discourse. Oxford UniversityPress (1997).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
He, D., Ritchie, G., Lee, J. (2000). Resolving References to Graphical Objects in Multimodal Queries by Constraint Satisfaction. In: Tan, T., Shi, Y., Gao, W. (eds) Advances in Multimodal Interfaces — ICMI 2000. ICMI 2000. Lecture Notes in Computer Science, vol 1948. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-40063-X_2
Download citation
DOI: https://doi.org/10.1007/3-540-40063-X_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41180-2
Online ISBN: 978-3-540-40063-9
eBook Packages: Springer Book Archive