research-article

StARe: Gaze-Assisted Face-to-Face Communication in Augmented Reality

Authors:

Yasmeen Abdrabou,

Augusto Esteves,

Stefanie Meitner,

Florian AltAuthors Info & Claims

ETRA '20 Adjunct: ACM Symposium on Eye Tracking Research and Applications

Article No.: 14, Pages 1 - 5

https://doi.org/10.1145/3379157.3388930

Published: 02 June 2020 Publication History

Abstract

This research explores the use of eye-tracking during Augmented Reality (AR) - supported conversations. In this scenario, users can obtain information that supports the conversation, without augmentations distracting the actual conversation.We propose using gaze that allows users to gradually reveal information on demand. Information is indicated around user’s head, which becomes fully visible when other’s visual attention explicitly falls upon the area. We describe the design of such an AR UI and present an evaluation of the feasibility of the concept. Results show that despite gaze inaccuracies, users were positive about augmenting their conversations with contextual information and gaze interactivity. We provide insights into the trade-offs between focusing on the task at hand (i.e., the conversation), and consuming AR information. These findings are useful for future use cases of eye based AR interactions by contributing to a better understanding of the intricate balance between informative AR and information overload.

Supplementary Material

MP4 File (a14-rivu-supplement.mp4)

Download
102.58 MB

References

[1]

Antti Ajanki, Mark Billinghurst, Hannes Gamper, Toni Järvenpää, Melih Kandemir, Samuel Kaski, Markus Koskela, Mikko Kurimo, Jorma Laaksonen, Kai Puolamäki, 2011. An augmented reality interface to contextual information. Virtual reality 15, 2-3 (2011), 161–173.

[2]

Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, and Blair MacIntyre. 2001. Recent Advances in Augmented Reality. IEEE Comput. Graph. Appl. 21, 6 (Nov. 2001), 34–47. https://doi.org/10.1109/38.963459

Digital Library

[3]

Blaine Bell, Steven Feiner, and Tobias Höllerer. 2001. View Management for Virtual and Augmented Reality(UIST ’01). ACM, New York, NY, USA, 101–110. https://doi.org/10.1145/502348.502363

[4]

Richard A Bolt. 1981. Gaze-orchestrated dynamic windows. In ACM SIGGRAPH Computer Graphics, Vol. 15. ACM, 109–119.

[5]

Rakesh D Desale and Vandana S Ahire. 2013. A study on wearable gestural interface–a sixthsense technology. IOSR Journal of Computer Engineering (IOSR-JCE) e-ISSN (2013), 2278–0661.

[6]

Steven Feiner, Blair MacIntyre, Marcus Haupt, and Eliot Solomon. 1993. Windows on the World: 2D Windows for 3D Augmented Reality(UIST ’93). ACM, New York, NY, USA, 145–155. https://doi.org/10.1145/168642.168657

[7]

Bernardo A Huberman and Fang Wu. 2007. The economics of attention: maximizing user value in information-rich environments. In Proceedings of the 1st international workshop on Data mining and audience intelligence for advertising. ACM, 16–20.

Digital Library

[8]

Yoshio Ishiguro and Jun Rekimoto. 2011. Peripheral Vision Annotation: Noninterference Information Presentation Method for Mobile Augmented Reality(AH ’11). ACM, New York, NY, USA, Article 8, 5 pages. https://doi.org/10.1145/1959826.1959834

[9]

Robert JK Jacob. 1993. Eye movement-based human-computer interaction techniques: Toward non-command interfaces. Advances in human-computer interaction 4 (1993), 151–190.

[10]

Robert J. K. Jacob. 1990. What You Look at is What You Get: Eye Movement-based Interaction Techniques(CHI ’90). ACM, New York, NY, USA, 11–18. https://doi.org/10.1145/97243.97246

[11]

Simon Julier, Yohan Baillot, Dennis Brown, and Marco Lanzagorta. 2002. Information Filtering for Mobile Augmented Reality. IEEE Comput. Graph. Appl. 22, 5 (Sept. 2002), 12–15. https://doi.org/10.1109/MCG.2002.1028721

Digital Library

[12]

Jens Keil, Michael Zoellner, Timo Engelke, Folker Wientapper, and Michael Schmitt. 2013. Controlling and filtering information density with spatial interaction techniques via handheld augmented reality. In International Conference on Virtual, Augmented and Mixed Reality. Springer, 49–57.

[13]

Mirae Kim, Min Kyung Lee, and Laura Dabbish. 2015. Shop-i: Gaze based Interaction in the Physical World for In-Store Social Shopping Experience. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems(CHI EA ’15). Association for Computing Machinery, Seoul, Republic of Korea, 1253–1258. https://doi.org/10.1145/2702613.2732797

Digital Library

[14]

Mikko Kytö, Barrett Ens, Thammathip Piumsomboon, Gun A. Lee, and Mark Billinghurst. 2018. Pinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality(CHI ’18). ACM, New York, NY, USA, Article 81, 14 pages. https://doi.org/10.1145/3173574.3173655

Digital Library

[15]

Ann McNamara, Katherine Boyd, David Oh, Ryan Sharpe, and Annie Suther. 2018. Using Eye Tracking to Improve Information Retrieval in Virtual Reality. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). 242–243. https://doi.org/10.1109/ISMAR-Adjunct.2018.00076 ISSN: null.

[16]

Ann McNamara and Chethna Kabeerdoss. 2016. Mobile Augmented Reality: Placing Labels Based on Gaze Position. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). 36–37. https://doi.org/10.1109/ISMAR-Adjunct.2016.0033 ISSN: null.

[17]

Jakob Nielsen. 1993. Noncommand User Interfaces. Commun. ACM 36, 4 (April 1993), 83–99. https://doi.org/10.1145/255950.153582

Digital Library

[18]

Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards Foveated Rendering for Gaze-Tracked Virtual Reality. ACM Trans. Graph. 35, 6, Article Article 179 (Nov. 2016), 12 pages. https://doi.org/10.1145/2980179.2980246

Digital Library

[19]

Marcus Tönnis and Gudrun Klinker. 2014. Boundary Conditions for Information Visualization with Respect to the User’s Gaze(AH ’14). ACM, New York, NY, USA, Article 44, 8 pages. https://doi.org/10.1145/2582051.2582095

[20]

Roel Vertegaal 2003. Attentive user interfaces. Commun. ACM 46, 3 (2003), 30–33.

Digital Library

[21]

Sean White and Steven Feiner. 2009. SiteLens: Situated Visualization Techniques for Urban Site Visits(CHI ’09). ACM, New York, NY, USA, 1117–1120. https://doi.org/10.1145/1518701.1518871

Cited By

Cools RVenema REsteves ASimeone A(2024)The Impact of Near-Future Mixed Reality Contact Lenses on Users' Lives via an Immersive Speculative Enactment and Focus GroupsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682099(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682099
Pohl H(2024)Body-Based Augmented Reality Feedback During ConversationsProceedings of the ACM on Human-Computer Interaction10.1145/36764918:MHCI(1-22)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676491
Lee JWang JBrown EChu LRodriguez SFroehlich J(2024)GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642230(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642230
Show More Cited By

Recommendations

Gaze-Hand Alignment: Combining Eye Gaze and Mid-Air Pointing for Interacting with Menus in Augmented Reality
ETRA

Gaze and freehand gestures suit Augmented Reality as users can interact with objects at a distance without need for a separate input device. We propose Gaze-Hand Alignment as a novel multimodal selection principle, defined by concurrent use of both gaze ...
Advantages of eye-gaze over head-gaze-based selection in virtual and augmented reality under varying field of views
COGAIN '18: Proceedings of the Workshop on Communication by Gaze Interaction

The current best practice for hands-free selection using Virtual and Augmented Reality (VR/AR) head-mounted displays is to use head-gaze for aiming and dwell-time or clicking for triggering the selection. There is an observable trend for new VR and AR ...
Looking for Info: Evaluation of Gaze Based Information Retrieval in Augmented Reality
Human-Computer Interaction – INTERACT 2021
Abstract
This paper presents the results of an empirical study and a real-world deployment of a gaze-adaptive UI for Augmented Reality (AR). AR introduces an attention dilemma between focusing on the reality vs. on AR content. Past work suggested eye gaze ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '20 Adjunct: ACM Symposium on Eye Tracking Research and Applications

June 2020

200 pages

ISBN:9781450371353

DOI:10.1145/3379157

Editors:
Andreas Bulling
University of Stuttgart, Germany
,
Anke Huckauf
Ulm University, Germany
,
Eakta Jain
University of Florida, USA
,
Ralph Radach
University of Wuppertal, Germany
,
Daniel Weiskopf
University of Stuttgart, Germany

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Studienstiftung des Deutschen Volkes

Conference

ETRA '20

Sponsor:

ETRA '20: 2020 Symposium on Eye Tracking Research and Applications

June 2 - 5, 2020

Stuttgart, Germany

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
509
Total Downloads

Downloads (Last 12 months)114
Downloads (Last 6 weeks)12

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cools RVenema REsteves ASimeone A(2024)The Impact of Near-Future Mixed Reality Contact Lenses on Users' Lives via an Immersive Speculative Enactment and Focus GroupsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682099(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682099
Pohl H(2024)Body-Based Augmented Reality Feedback During ConversationsProceedings of the ACM on Human-Computer Interaction10.1145/36764918:MHCI(1-22)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676491
Lee JWang JBrown EChu LRodriguez SFroehlich J(2024)GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642230(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642230
Huang YChen GXu JZhang MYang LPei BZhang HDong LWang YWang LQiao Y(2024)EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02084(22072-22086)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02084
Asadi AAppiah JMuntaka SKropczynski J(2024)Actions, Not Apps: Toward Using LLMs to Reshape Context Aware Interactions in Mixed Reality SystemsHCI International 2024 Posters10.1007/978-3-031-62110-9_17(166-176)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-62110-9_17
Huang YXu LZhou YChen QWu ZFeng LFan M(2023)Toward Leveraging Augmented Reality (AR) for Enhancing Remote Intergenerational Communication in Cooking ScenariosProceedings of the Eleventh International Symposium of Chinese CHI10.1145/3629606.3629658(491-496)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3629606.3629658
Wecker DYigitbas E(2023)Minimizing Eye Movements and Distractions in Head-Mounted Augmented Reality through Eye-Gaze AdaptivenessProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3614524(1-12)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3614524
Schön DKosch TMüller FSchmitz MGünther SBommhardt LMühlhäuser M(2023)Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581461(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581461
Cai RJanaka NZhao SSun M(2023)ParaGlassMenu: Towards Social-Friendly Subtle Interactions in ConversationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581065(1-21)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581065
Namnakani OAbdrabou YGrizou JEsteves AKhamis M(2023)Comparing Dwell time, Pursuits and Gaze Gestures for Gaze Interaction on Handheld Mobile DevicesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580871(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580871
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents