Abstract
Following the ecological approach to visual perception, this paper presents a framework that emphasizes the role of vision on referring actions. In particular, affordances are utilized to explain gestures variability in a multimodal human-computer interaction. Such a proposal is consistent with empirical findings obtained in different simulation studies showing how referring gestures are determined by the mutuality of information coming from the target and the set of movements available to the speaker. A prototype that follows anthropomorphic perceptual principles to analyze gestures has been developed and tested in preliminary computational validations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
De Angeli, A., Gerbino, W., Petrelli, D., Cassano, G.: Visual display, pointing and natural language: The power of multimodal interaction. In: Proceedings of the Working Conference on Advanced Visual Interface AVI’98, L’Aquila, Italy, May 1998. ACM Press (1998), 164–173
Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton Mifflin Boston (1979).
Maybury, M.T.: Intelligent Multimdedia Interfaces. Cambridge Mass. MIT Press (1993).
Oviatt, S., De Angeli. A, Kuhn, K.: Integration and sysnchronisation of input modes during multimodal human-computer interaction. In: Proceedings of the CHI’97 Conference, New York ACM Press (1997), 415–422.
Wolff, F., De Angeli, A., Romary, L.: Acting on a visual world: The role of perception in multimodal HCI. In: Proceedings of the 1998Workshop on Representations for Multi-Modal Human-Computer Interaction. AAAI Press (1998).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
De Angeli, A., Wolff, F., Romary, L., Gerbino, W. (1999). The Ecological Approach to Multimodal System Design. In: Braffort, A., Gherbi, R., Gibet, S., Teil, D., Richardson, J. (eds) Gesture-Based Communication in Human-Computer Interaction. GW 1999. Lecture Notes in Computer Science(), vol 1739. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46616-9_4
Download citation
DOI: https://doi.org/10.1007/3-540-46616-9_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66935-7
Online ISBN: 978-3-540-46616-1
eBook Packages: Springer Book Archive