Article

Furhat: a back-projected human-like robot head for multiparty human-machine interaction

Authors:

Samer Al Moubayed,

Gabriel Skantze,

Björn GranströmAuthors Info & Claims

COST'11: Proceedings of the 2011 international conference on Cognitive Behavioural Systems

Pages 114 - 130

https://doi.org/10.1007/978-3-642-34584-5_9

Published: 21 February 2011 Publication History

Abstract

In this chapter, we first present a summary of findings from two previous studies on the limitations of using flat displays with embodied conversational agents (ECAs) in the contexts of face-to-face human-agent interaction. We then motivate the need for a three dimensional display of faces to guarantee accurate delivery of gaze and directional movements and present Furhat, a novel, simple, highly effective, and human-like back-projected robot head that utilizes computer animation to deliver facial movements, and is equipped with a pan-tilt neck. After presenting a detailed summary on why and how Furhat was built, we discuss the advantages of using optically projected animated agents for interaction. We discuss using such agents in terms of situatedness, environment, context awareness, and social, human-like face-to-face interaction with robots where subtle nonverbal and social facial signals can be communicated. At the end of the chapter, we present a recent application of Furhat as a multimodal multiparty interaction system that was presented at the London Science Museum as part of a robot festival,. We conclude the paper by discussing future developments, applications and opportunities of this technology.

References

[1]

Dominik, Z.: Who did actually invent the word robotand what does it mean? The Karel Čapek website, http://capek.misto.cz/english/robot.html (retrieved December 10, 2011)

[2]

Summerfield, Q.: Lipreading and audio-visual speech perception. Philosophical Transactions: Biological Sciences 335(1273), 71-78 (1992)

[3]

Al Moubayed, S., Beskow, J.: Effects of Visual Prominence Cues on Speech Intelligibility. In: Proceedings of Auditory-Visual Speech Processing, AVSP 2009, Norwich, England (2009)

[4]

Argyle, M., Cook, M.: Gaze and mutual gaze. Cambridge University Press (1976)

[5]

Kleinke, C. L.: Gaze and eye contact: a research review. Psychological Bulletin 100, 78-100 (1986)

[6]

Ekman, P., Friesen, W. V.: Unmasking the face: A guide to recognizing emotions from facial clues. Malor Books (2003) ISBN: 978-1883536367

[7]

Shinozawa, K., Naya, F., Yamato, J., Kogure, K.: Differences in effect of robot and screen agent recommendations on human decision-making. International Journal of Human Computer Studies 62(2), 267-279 (2005)

Digital Library

[8]

Mori, M.: Bukimi no tani.:The uncanny valley (K. F. MacDorman & T. Minato, Trans.). Energy 7(4), 33-35 (1970) (Originally in Japanese)

[9]

Gockley, R., Simmons, J., Wang, D., Busquets, C., DiSalvo, K., Caffrey, S., Rosenthal, J., Mink, S., Thomas, W., Adams, T., Lauducci, M., Bugajska, D., Perzanowski, Schultz, A.: Grace and George: Social Robots at AAAI. In: Proceedings of AAAI 2004, Mobile Robot Competition Workshop, pp. 15-20. AAAI Press (2004)

[10]

Edlund, J., Al Moubayed, S., Beskow, J.: The Mona Lisa Gaze Effect as an Objective Metric for Perceived Cospatiality. In: Vilhjálmsson, H. H., Kopp, S., Marsella, S., Thórisson, K. R. (eds.) IVA 2011. LNCS (LNAI), vol. 6895, pp. 439-440. Springer, Heidelberg (2011)

Digital Library

[11]

Todorovi, D.: Geometrical basis of perception of gaze direction. Vision Research 45(21), 3549-3562 (2006)

[12]

Raskar, R., Welch, G., Low, K.-L., Bandyopadhyay, D.: Shader lamps: animating real objects with image-based illumination. In: Proc. of the 12th Eurographics Workshop on Rendering Techniques, pp. 89-102 (2001)

Digital Library

[13]

Lincoln, P., Welch, G., Nashel, A., Ilie, A., State, A., Fuchs, H.: Animatronic shader lamps avatars. In: Proc. of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2009). IEEE Computer Society, Washington, DC (2009)

Digital Library

[14]

Al Moubayed, S., Edlund, J., Beskow, J.: Taming Mona Lisa: Communicating gaze faithfully in 2D and 3D facial projections. ACM Trans. Interact. Intell. Syst. 1(2), Article 11, 25 pages (2012)

Digital Library

[15]

Al Moubayed, S., Skantze, G.: Turn-taking Control Using Gaze in Multiparty Human-Computer Dialogue: Effects of 2D and 3D Displays. In: Proceedings of the International Conference on Auditory-Visual Speech Processing AVSP, Florence, Italy (2011)

[16]

Al Moubayed, S., Beskow, J., Edlund, J., Granström, B., House, D.: Animated Faces for Robotic Heads: Gaze and Beyond. In: Esposito, A., Vinciarelli, A., Vicsi, K., Pelachaud, C., Nijholt, A. (eds.) Communication and Enactment 2010. LNCS, vol. 6800, pp. 19-35. Springer, Heidelberg (2011)

Digital Library

[17]

Beskow, J.: Talking heads - Models and applications for multimodal speech synthesis. Doctoral dissertation, KTH (2003)

[18]

Beskow, J.: Animation of talking agents. In: Benoit, C., Campbel, R. (eds.) Proc of ESCA Workshop on Audio-Visual Speech Processing, Rhodes, Greece, pp. 149-152 (1997)

[19]

Granström, B., House, D.: Modeling and evaluating verbal and non-verbal communication in talking animated interface agents. In: Dybkjaer, l., Hemsen, H., Minker, W. (eds.) Evaluation of Text and Speech Systems, pp. 65-98. Springer (2007)

[20]

Al Moubayed, S., Beskow, J., Granström, B.: Auditory-Visual Prominence: From Intelligibility to Behavior. Journal on Multimodal User Interfaces 3(4), 299-311 (2010)

[21]

Brouwer, D. M., Bennik, J., Leideman, J., Soemers, H. M. J. R., Stramigioli, S.: Mechatronic Design of a Fast and Long Range 4 Degrees of Freedom Humanoid Neck. In: Proceedings of ICRA, Kobe, Japan, ThB8.2, pp. 574-579 (2009)

Digital Library

[22]

Harel, D.: Statecharts: A visual formalism for complex systems. Science of Computer Programming 8(3), 231-274 (1987)

Digital Library

[23]

Blackwell, R. D., Hensel, J. S., Sternthal, B.: Pupil dilation: What does it measure? Journal of Advertising Research 10, 15-18 (1970)

[24]

Nishino, K., Nayar, S. K.: Corneal Imaging System: Environment from Eyes. Int. J. Comput. Vision 70(1), 23-40 (2006).

Digital Library

Cited By

Cruz-Sandoval DTentori MFavela J(2024)A Framework to Design Engaging Interactions in Socially Assistive Robots to Mitigate Dementia-Related SymptomsACM Transactions on Human-Robot Interaction10.1145/370088914:1(1-25)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3700889
Calvo-Barajas NAkkuzu ACastellano G(2024)Balancing Human Likeness in Social Robots: Impact on Children’s Lexical Alignment and Self-disclosure for Trust AssessmentACM Transactions on Human-Robot Interaction10.1145/365906213:4(1-27)Online publication date: 23-Oct-2024
https://dl.acm.org/doi/10.1145/3659062
Kamelabad AEngwall OSkantze G(2024)Conformity and Trust in Multi-party vs. Individual Human-Robot InteractionProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673954(1-9)Online publication date: 16-Sep-2024
https://dl.acm.org/doi/10.1145/3652988.3673954
Show More Cited By

Recommendations

Taming Mona Lisa: Communicating gaze faithfully in 2D and 3D facial projections

The perception of gaze plays a crucial role in human-human interaction. Gaze has been shown to matter for a number of aspects of communication and dialogue, especially for managing the flow of the dialogue and participant attention, for deictic ...
Spontaneous spoken dialogues with the furhat human-like robot head
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction

Furhat [1] is a robot head that deploys a back-projected animated face that is realistic and human-like in anatomy. Furhat relies on a state-of-the-art facial animation architecture allowing accurate synchronized lip movements with speech, and the ...
Multimodal multiparty social interaction with the furhat head
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interaction

We will show in this demonstrator an advanced multimodal and multiparty spoken conversational system using Furhat, a robot head based on projected facial animation. Furhat is a human-like interface that utilizes facial animation for physical robot heads ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

COST'11: Proceedings of the 2011 international conference on Cognitive Behavioural Systems

February 2011

448 pages

ISBN:9783642345838

Editors:
Anna Esposito
Department of Psychology, and IIASS, Seconda Università degli Studi di Napoli, Italy
,
Antonietta M. Esposito
Department of Psychology, and IIASS, Istituto Nazionale di Geofisica e Vulcanologia, sezione di Napoli Osservatorio Vesuviano, Napoli, Italy
,
Alessandro Vinciarelli
School of Computing Science, University of Glasgow, Glasgow, UK
,
Rüdiger Hoffmann
Laboratory of Acoustics and Speech Communication, Technische Universität Dresden, Dresden, Germany
,
Vincent C. Müller
Dept. of Humanities and Social Sciences, Anatolia College/ACT, Pylaia, Greece

Sponsors

SSPNET: Social Signal Processing Network
International Institute for Advanced Scientific Studies "E.R. Caianiello": International Institute for Advanced Scientific Studies "E.R. Caianiello"
European COST Action 2102: European COST Action 2102
Second University of Naples: Second University of Naples
SERN: Società Italiana Reti Neuroniche

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 21 February 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cruz-Sandoval DTentori MFavela J(2024)A Framework to Design Engaging Interactions in Socially Assistive Robots to Mitigate Dementia-Related SymptomsACM Transactions on Human-Robot Interaction10.1145/370088914:1(1-25)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3700889
Calvo-Barajas NAkkuzu ACastellano G(2024)Balancing Human Likeness in Social Robots: Impact on Children’s Lexical Alignment and Self-disclosure for Trust AssessmentACM Transactions on Human-Robot Interaction10.1145/365906213:4(1-27)Online publication date: 23-Oct-2024
https://dl.acm.org/doi/10.1145/3659062
Kamelabad AEngwall OSkantze G(2024)Conformity and Trust in Multi-party vs. Individual Human-Robot InteractionProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673954(1-9)Online publication date: 16-Sep-2024
https://dl.acm.org/doi/10.1145/3652988.3673954
Francese RCiobanu MClemente ETortora G(2024)Design of a Multimodal Robot-Based Conversational Interface: A Case Study with FURHATHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76803-3_17(299-311)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-76803-3_17
Beelen TOrdelman RTruong KEvers VHuibers T(2024)A Conversational Robot for Children’s Access to a Cultural Heritage Multimedia ArchiveAdvances in Information Retrieval10.1007/978-3-031-56069-9_11(144-151)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56069-9_11
Pasquali DKothig AAroyo AMuñoz Cadorna JDautenhahn KBencetti SFrancesco RSciutti A(2023)That's not a Good Idea: A Robot Changes Your Behavior Against Social EngineeringProceedings of the 11th International Conference on Human-Agent Interaction10.1145/3623809.3623879(63-71)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3623809.3623879
Rasouli SJohnston LYuen JGhafurian MFoster LDautenhahn K(2023)Co-Design of a Robotic Mental Well-Being Coach to Help University Students Manage Public Speaking AnxietyProceedings of the 11th International Conference on Human-Agent Interaction10.1145/3623809.3623872(200-208)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3623809.3623872
Serfaty GBarnard VSalisbury J(2023)Generative Facial Expressions and Eye Gaze Behavior from Prompts for Multi-Human-Robot InteractionAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586182.3616623(1-3)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586182.3616623
Kamelabad ASkantze GCastellano GRiek LCakmak MLeite I(2023)I Learn Better Alone! Collaborative and Individual Word Learning With a Child and Adult RobotProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3577004(368-377)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3568162.3577004
Beelen TVelner ETruong KOrdelman RHuibers TEvers V(2023)Children’s Trust in Robots and the Information They ProvideExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585801(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585801
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents