research-article

LiveMask: a telepresence surrogate system with a face-shaped screen for supporting nonverbal communication

Authors:

Yoshio Ishiguro,

Jun RekimotoAuthors Info & Claims

AVI '12: Proceedings of the International Working Conference on Advanced Visual Interfaces

Pages 394 - 397

https://doi.org/10.1145/2254556.2254632

Published: 21 May 2012 Publication History

Abstract

We propose a telepresence system with a real human face-shaped screen. This system tracks the remote user's face and extracts the head motion and the face image. The face-shaped screen moves along three degree-of-freedom (DOF) by reflecting the user's head gestures. As the face-shaped screen is molded based on the 3D-shape scan data of the user, the projected image is accurate even when it is seen from different angles. We expect this system can accurately convey the user's nonverbal communication, in particular the user's gaze direction in 3D space that is not correctly transmitted by using a 2D screen (which is known as "the Mona Lisa effect"). To evaluate how this system can contribute to the communication, we conducted three experiments. The first one examines the blind angle of a face-shaped screen and a flat screen, and compares the ease with which users can distinguish facial expressions. The second one evaluates how the direction in which the remote user's face points can be correctly transmitted. The third experiment evaluates how the gaze direction can be correctly transmitted. We found that the recognizable angles of the face-shaped screen were larger, and that the recognition of the head directions was better than on a flat 2D screen. More importantly, we found that the face-shaped screen accurately conveyed the gaze direction, resolving the problem of the Mona Lisa effect.

References

[1]

AnyBots https://www.anybots.com.

[2]

Cisco Telepresence. http://www.cisco.com/en/US/products/ps7060/index.html.

[3]

HP Halo. http://hphalo.org/.

[4]

Hydra. http://www.billbuxton.com/hydraNarrative.htm.

[5]

Seeing Machine faceAPI. http://www.seeingmachines.com/product/faceapi/.

[6]

Talking Head Projection. http://www.naimark.net/projects/head.html.

[7]

S. O. Adalgeirsson and C. Breazeal. Mebot: a robotic platform for socially embodied presence. Proc. HRI' 10, 15--22, 2010.

Digital Library

[8]

M. Argyle, L. Lefebvre, and M. Cook. The meaning of five patterns of gaze. European Journal of Social Psychology, 4(2), 125--136, 1974.

[9]

J. Beskow and S. Al Moubayed. Perception of gaze direction in 2d and 3d facial projections. Proc. FAA '10, 24, 2010.

Digital Library

[10]

F. Delaunay, J. De Greeff, and T. Belpaeme. A study of a retro-projected robotic face and its effectiveness for gaze reading by humans. Proc. HRI'10, 39--44, 2010.

Digital Library

[11]

S. A. M. Jens Edlund and B. Jonas. The mona lisa gaze effect as an objective metric for perceived cospatiality. Proc. IVA'11, 439--440, 2011.

Digital Library

[12]

A. Kendon. Some functions of gaze-direction in social interaction. Acta Psychologica, 26(1), 22--63, 1967.

[13]

M. K. Lee and L. Takayama. "Now, i have a body": uses and social norms for mobile remote presence in the workplace. Proc. CHI'11, 33--42, 2011.

Digital Library

[14]

H. Nagahara, Y. Yagi, and M. Yachida. Wide field of view head mounted display for tele-presence with an omnidirectional image sensor. CVPR Workshop, 7:86, 2003.

[15]

H. Nakanishi, K. Kato, and H. Ishiguro. Zoom cameras and movable displays enhance social telepresence. Proc. CHI'11, 63--72, 2011.

Digital Library

[16]

E. Paulos and J. Canny. Social tele-embodiment: Understanding presence. Auton. Robots, 87--95, 2001.

Digital Library

[17]

D. Sakamoto, T. Kanda, T. Ono, H. Ishiguro, and N. Hagita. Android as a telecommunication medium with a human-like presence. Proc. HRI'07, 193--200, 2007.

Digital Library

Cited By

Wang XZhang WFu H(2024)A3RT: Attention-Aware AR Teleconferencing with Life-Size 2.5D Video Avatars2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00044(211-221)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00044
Faridan MKumari BSuzuki R(2023)ChameleonControl: Teleoperating Real Human Surrogates through Mixed Reality Gestural Guidance for Remote Hands-on ClassroomsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581381(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581381
Sakashita MZhang RLi XKim HRusso MZhang CJung MGuimbretière F(2023)ReMotion: Supporting Remote Collaboration in Open Space with Automatic Robotic EmbodimentProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580699(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580699
Show More Cited By

Index Terms

LiveMask: a telepresence surrogate system with a face-shaped screen for supporting nonverbal communication
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Computer supported cooperative work
2. Social and professional topics
  1. Professional topics
    1. Computing and business
      1. Computer supported cooperative work

Recommendations

Ma petite chérie: what are you looking at?: a small telepresence system to support remote collaborative work for intimate communication
AH '12: Proceedings of the 3rd Augmented Human International Conference

We present a telepresence system with a reduced scale face-shaped display for supporting intimate telecommunication. In our previous work, we have developed a real-size face shaped display that tracks and reproduces the remote user's head motion and ...
Pursuits: eye-based interaction with moving targets
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Eye-based interaction has commonly been based on estimation of eye gaze direction, to locate objects for interaction. We introduce Pursuits, a novel and very different eye tracking method that instead is based on following the trajectory of eye movement ...
EFG-Net: A Unified Framework for Estimating Eye Gaze and Face Gaze Simultaneously
Pattern Recognition and Computer Vision
Abstract
Gaze is of vital importance for understanding human purpose and intention. Recent works have gained tremendous progress in appearance-based gaze estimation. However, all these works deal with eye gaze estimation or face gaze estimation separately, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AVI '12: Proceedings of the International Working Conference on Advanced Visual Interfaces

May 2012

846 pages

ISBN:9781450312875

DOI:10.1145/2254556

Editors:
Genny Tortora
Università di Salerno, Italy
,
Stefano Levialdi
Sapienza Università di Roma, Italy
,
Maurizio Tucci
Università di Salerno, Italy

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Consulta Umbria SRL
University of Salerno: University of Salerno

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 May 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AVI'12

Sponsor:

University of Salerno

AVI'12: International Working Conference on Advanced Visual Interfaces

May 21 - 25, 2012

Capri Island, Italy

Acceptance Rates

Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
442
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)3

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang XZhang WFu H(2024)A3RT: Attention-Aware AR Teleconferencing with Life-Size 2.5D Video Avatars2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00044(211-221)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00044
Faridan MKumari BSuzuki R(2023)ChameleonControl: Teleoperating Real Human Surrogates through Mixed Reality Gestural Guidance for Remote Hands-on ClassroomsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581381(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581381
Sakashita MZhang RLi XKim HRusso MZhang CJung MGuimbretière F(2023)ReMotion: Supporting Remote Collaboration in Open Space with Automatic Robotic EmbodimentProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580699(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580699
Sakashita MRicci EArora JGuimbretière F(2022)RemoteCoDe: Robotic Embodiment for Enhancing Peripheral Awareness in Remote Collaboration TasksProceedings of the ACM on Human-Computer Interaction10.1145/35129106:CSCW1(1-22)Online publication date: 7-Apr-2022
https://dl.acm.org/doi/10.1145/3512910
Ichii TMitake HHasegawa S(2020)TEllipsoid: Ellipsoidal Display for Videoconference System Transmitting Accurate Gaze Direction2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW50115.2020.00274(847-848)Online publication date: Mar-2020
https://doi.org/10.1109/VRW50115.2020.00274
Ichii TMitake HHasegawa S(2020)TEllipsoid: Ellipsoidal Display for Videoconference System Transmitting Accurate Gaze Direction2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR46266.2020.1580098058128(775-781)Online publication date: Mar-2020
https://doi.org/10.1109/VR46266.2020.1580098058128
Kobayashi KKomuro TKagawa KKawahito S(2020)Transmission of correct gaze direction in video conferencing using screen-embedded camerasMultimedia Tools and Applications10.1007/s11042-020-09758-wOnline publication date: 19-Sep-2020
https://doi.org/10.1007/s11042-020-09758-w
Ishiguro YTakeda KJanssen CDonker SChuang LJu W(2019)LeadingDisplayProceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings10.1145/3349263.3351317(405-409)Online publication date: 21-Sep-2019
https://dl.acm.org/doi/10.1145/3349263.3351317
Myodo EXu JTasaka KYanagihara HSakazawa S(2018)[Invited Paper] Issues and Solutions to Informal Communication in Working from Home Using a Telepresence RobotITE Transactions on Media Technology and Applications10.3169/mta.6.306:1(30-45)Online publication date: 2018
https://doi.org/10.3169/mta.6.30
Shiro KOkada AMiyaki TRekimoto JImai MNorman TSklar EKomatsu T(2018)OmniGazeProceedings of the 6th International Conference on Human-Agent Interaction10.1145/3284432.3284439(176-183)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3284432.3284439
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten