article

Eye movements as indices for the utility of life-like interface agents: A pilot study

Authors:

Helmut Prendinger,

Mitsuru IshizukaAuthors Info & Claims

Interacting with Computers, Volume 19, Issue 2

Pages 281 - 292

https://doi.org/10.1016/j.intcom.2006.10.004

Published: 01 March 2007 Publication History

Abstract

We motivate an approach to evaluating the utility of life-like interface agents that is based on human eye movements rather than questionnaires. An eye tracker is employed to obtain quantitative evidence of a user's focus of attention without distracting from the primary task. The salient feature of our evaluation strategy is that it allows us to measure important properties of a user's interaction experience on a moment-by-moment basis in addition to a cumulative (spatial) analysis of the user's areas of interest. We describe a pilot study in which we compare attending behavior of subjects watching the presentation of a computer-generated apartment layout and visualization augmented by three types of media: an animated agent, a text box, and speech only. The investigation of eye movements revealed that deictic gestures performed by the agent are more effective in directing the attentional focus of subjects to relevant interface objects than the media used in the two control conditions, at a slight cost of distracting the user from visual inspection of the object of reference. The results also demonstrate that the presence of an interface agent seemingly triggers natural and social interaction protocols of human users.

References

[1]

The PPP Persona: A multipurpose animated presentation agent. In: Proceedings Advanced Visual Interfaces (AVI-96), ACM Press, New York. pp. 245-247.

Digital Library

[2]

The role of emotion in believable agents. Communications of the ACM. v37 i7. 122-125.

Digital Library

[3]

. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (Eds.), Embodied Conversational Agents, The MIT Press, Cambridge, MA.

[4]

The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology. v6. 84-107.

[5]

Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Educational Psychology. v94 i2. 428-434.

[6]

The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies. i52. 1-22.

Digital Library

[7]

Eye Tracking Methodology: Theory and Practice. Springer, London, UK.

Digital Library

[8]

Faraday, P., Sutcliffe, A., 1996. An empirical study of attending and comprehending multimedia presentations. In: Proceedings of ACM Multimedia 96, Boston MA, pp. 265-275.

Digital Library

[9]

Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics. v24. 631-645.

[10]

Pupillometrics: A method of studying mental, emotional and sensory processes. In: Greenfield, N., Sternbach, R. (Eds.), Handbook of Psychophysiology, Holt, Rinehart & Winston, New York. pp. 491-531.

[11]

Attention effect of animated character. In: Proceedings Human-Computer Interaction (INTERACT-03), IOS Press. pp. 423-430.

[12]

Hottolink, 2002. Hottolink Inc. Available from: http://www.hottolink.com.

[13]

The use of eye movements in human-computer interaction techniques: What you look at is what you get. ACM Transactions on Information Systems. v9 i3. 152-169.

Digital Library

[14]

This computer responds to user frustration: Theory, design, and results. Interacting with Computers. v14. 119-140.

[15]

The Persona effect: Affective impact of animated pedagogical agents. In: Proceedings of CHI-97, ACM Press, New York. pp. 359-366.

Digital Library

[16]

Autonomic specificity and emotion. In: Davidson, R.J., Scherer, K.R., Goldsmith, H.H. (Eds.), Handbook of Affective Sciences, Oxford University Press, Oxford. pp. 212-224.

[17]

Effective attention allocation behavior and its measurement: A preliminary study. Interacting with Computers. v16. 1195-1210.

[18]

Experimental assessment of the effectiveness of synthetic personae for multi-modal e-retail applications. In: Proceedings 4th International Conference on Autonomous Agents (Agents'2000), ACM Press, New York. pp. 39-45.

Digital Library

[19]

Developing for Microsoft Agent. Microsoft Press, Redmond, WA.

Digital Library

[20]

Animated pedagogical agents in educational technology. Educational Technology. v44 i6. 23-30.

[21]

NAC, 2004. Image Technology. Available from: http://eyemark.jp.

[22]

Nakano, Y.I., Reinstein, G., Stocky, T., Cassell, J., 2003. Towards a model of face-to-face grounding. In: Proceedings of Association for Computational Linguistics (ACL-03). pp. 553-561.

Digital Library

[23]

Telling more than we know: Verbal reports on mental processes. Psychological Review. v84. 231-259.

[24]

Eye tracking as a new interface for image retrieval. British Telecommunications Technology Journal. v22 i3.

Digital Library

[25]

Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies. v59. 185-198.

Digital Library

[26]

Affective Computing. The MIT Press, Cambridge, MA.

[27]

. In: Prendinger, H., Ishizuka, M. (Eds.), Life-Like Characters. Tools, Affective Functions, and Applications. Cognitive Technologies, Springer Verlag, Berlin, Heidelberg.

Digital Library

[28]

The Empathic Companion: A character-based interface that addresses users' affective states. International Journal of Applied Artificial Intelligence. v19 i3. 267-285.

[29]

MPML: A markup language for controlling the behavior of life-like characters. Journal of Visual Languages and Computing. v15 i2. 183-203.

[30]

Understanding the effect of life-like interface agents through eye users' eye movements. In: Proceedings of Seventh International Conference on Multimodal Interfaces (ICMI-05), ACM Press, New York. pp. 108-115.

Digital Library

[31]

Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game. International Journal of Human-Computer Studies. v62 i2. 231-245.

Digital Library

[32]

Qu, L., Wang, N., Johnson, W.L., 2004. Pedagogical agents that interact with learners. In: AAMAS-04 Workshop on Balanced Perception and Action in ECAs, New York, NY, USA.

[33]

Reeves, B., Nass, C., 1998. The media equation. How People Treat Computers, Television and New Media Like Real People and Places. CSLI Publications, Center for the Study of Language and Information. Cambridge University Press.

Digital Library

[34]

Understanding visual influence in graph design through temporal and spatial eye movement characterisitics. Interacting with Computers. v16. 557-578.

[35]

A review of the development of embodied presentation agents and their application fields. In: Prendinger, H., Ishizuka, M. (Eds.), Life-like Characters. Tools, Affective Functions and Applications. Cognitive Technologies, Springer, Berlin, Heidelberg. pp. 377-404.

[36]

Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the Eye Tracking Research and Applications Symposium, ACM Press, New York. pp. 71-78.

Digital Library

[37]

Situated facial displays: Towards social interaction. In: Proceedings CHI 95 Conference, ACM Press, New York. pp. 450-455.

Digital Library

[38]

Tokyo Mansions, 2004. Tokyo Mansions. Available from: http://www.themansions.jp/.

[39]

Detection of user's confusion and surprise based on pupil dilation. The Japanese Journal of Ergonomics. v39 i4. 153-161.

[40]

The Persona Effect: How substantial is it?. In: Proceedings Human Computer Interaction (HCI-98), Springer, Berlin. pp. 53-66.

Digital Library

[41]

Physiological responses to different WEB page designs. International Journal of Human-Computer Studies. v59. 199-212.

Digital Library

[42]

Listen to your heart rate: Counting the cost of media quality. In: Paiva, A. (Ed.), Affective Interactions - Towards a New Generation of Computer Interfaces, Springer, Berlin, Heidelberg. pp. 9-20.

Digital Library

[43]

Witkowski, M., Arafa, Y., de Bruijn, O., 2001. Evaluating user reaction to character agent mediated displays using eye-tracking technology. In: Proceedings AISB-01 Symposium on Information Agents for Electronic Commerce. pp. 79-87.

Cited By

Volonte MAnaraky RKnijnenburg BDuchowski ABabu S(2019)Empirical Evaluation of the Interplay of Emotion and Visual Attention in Human-Virtual Human InteractionACM Symposium on Applied Perception 201910.1145/3343036.3343118(1-9)Online publication date: 19-Sep-2019
https://dl.acm.org/doi/10.1145/3343036.3343118
Diner SDoanay A(2017)The effects of multiple-pedagogical agents on learners academic success, motivation, and cognitive loadComputers & Education10.1016/j.compedu.2017.04.005111:C(74-100)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1016/j.compedu.2017.04.005
Dalzel-Job SNicol COberlander JRäihä KDuchowski A(2008)Comparing behavioural and self-report measures of engagement with an embodied conversational agentProceedings of the 2008 symposium on Eye tracking research & applications10.1145/1344471.1344493(83-85)Online publication date: 26-Mar-2008
https://dl.acm.org/doi/10.1145/1344471.1344493
Show More Cited By

Index Terms

Eye movements as indices for the utility of life-like interface agents: A pilot study
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
    2. Distributed artificial intelligence
      1. Intelligent agents
  2. Computer graphics
    1. Animation
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Understanding the effect of life-like interface agents through users' eye movements
ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

We motivate an approach to evaluating the utility of life-like interface agents that is based on human eye movements rather than questionnaires. An eye tracker is employed to obtain quantitative evidence of a user's focus of attention. The salient ...
A method to recognize eye movements based on uplift movement of skin
UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

Movements of human eyes provide useful insights for understanding human's physical and mental condition and ability, which has high attention in various industrial and academic fields. Although many methods for sensing eye movements have been proposed, ...
EyeFIX: An Interactive Visual Analytics Interface for Eye Movement Analysis
VINCI '21: Proceedings of the 14th International Symposium on Visual Information Communication and Interaction

Eye movements are closely related to the cognitive processes and often act as a window to the brain and mind. To facilitate a window to the brain using eye movements, we propose EyeFIX, an interactive visual analytics interface. Our interface currently ...

Comments

Information & Contributors

Information

Published In

cover image Interacting with Computers

Interacting with Computers Volume 19, Issue 2

March, 2007

170 pages

ISSN:0953-5438

EISSN:1873-7951

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2006.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 March 2007

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Volonte MAnaraky RKnijnenburg BDuchowski ABabu S(2019)Empirical Evaluation of the Interplay of Emotion and Visual Attention in Human-Virtual Human InteractionACM Symposium on Applied Perception 201910.1145/3343036.3343118(1-9)Online publication date: 19-Sep-2019
https://dl.acm.org/doi/10.1145/3343036.3343118
Diner SDoanay A(2017)The effects of multiple-pedagogical agents on learners academic success, motivation, and cognitive loadComputers & Education10.1016/j.compedu.2017.04.005111:C(74-100)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1016/j.compedu.2017.04.005
Dalzel-Job SNicol COberlander JRäihä KDuchowski A(2008)Comparing behavioural and self-report measures of engagement with an embodied conversational agentProceedings of the 2008 symposium on Eye tracking research & applications10.1145/1344471.1344493(83-85)Online publication date: 26-Mar-2008
https://dl.acm.org/doi/10.1145/1344471.1344493
Prendinger HEichner TAndré EIshizuka MInakage MLee NTscheligi MBernhaupt RNatkin S(2007)Gaze-based infotainment agentsProceedings of the international conference on Advances in computer entertainment technology10.1145/1255047.1255064(87-90)Online publication date: 13-Jun-2007
https://dl.acm.org/doi/10.1145/1255047.1255064

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents