Towards Computational Identification of Visual Attention on Interactive Tabletops
Pages 37 - 40
Abstract
There is a growing interest in the ability to detect where people are looking in real-time to support learning, collaboration, and efficiency. Here we present an overview of computational methods for accurately classifying the area of visual attention on a horizontal surface that we use to represent an interactive display (i.e. tabletop). We propose a new model that utilizes a neural network to estimate the area of visual attention, and provide a close examination of the factors that contribute to the accuracy of the model. Additionally, we discuss the use of this technique to model joint visual attention in collaboration. We achieved a mean classification accuracy of 75.75% with a standard deviation of 0.14 when data from four participants was used in training the model and then tested on the fifth participant. We also achieved a mean classification accuracy of 98.8% with 0.02 standard deviation when different amounts of overall data was used to test the model.
Supplementary Material
References
[1]
Shan Bao and Linda Ng Boyle. 2009. Age-related differences in visual scanning at median-divided highway intersections in rural areas. Accident Analysis & Prevention, Vol. 41, 1 (2009), 146--152.
[2]
Stéphanie Buisine, Guillaume Besacier, Améziane Aoussat, and Frederic Vernier. 2012. How do interactive tabletop systems influence collaboration? Computers in Human Behavior, Vol. 28, 1 (2012), 49 -- 59. https://doi.org/10.1016/j.chb.2011.08.010
[3]
Sheelagh Carpendale, Tobias Isenberg, Stacey D Scott, Uta Hinrichs, André Miede, Russel Kruger, Stefan Habelski, and Kori Inkpen. 2006. Collaborative interaction on large tabletop displays. In CONFERENCE SUPPLEMENT OF THE 2006 ACM CONFERENCE ON COMPUTER SUPPORTED COLLABORATIVE WORK (CSCW 2006, NOVEMBER 4--8, 2006, BANFF, ALBERTA, CANADA).
[4]
Demjén E, Abovsi V, and Tomori Z. 2011. Eye tracking using artificial neural networks for human computer interaction. Physiol Res, Vol. 60, 5 (2011), 841--844. https://www.ncbi.nlm.nih.gov/ /21812517
[5]
Amanda Harris, Jochen Rick, Victoria Bonnett, Nicola Yuill, Rowanne Fleck, Paul Marshall, and Yvonne Rogers. 2009. Around the Table: Are Multiple-Touch Surfaces Better than Single-Touch for Children's Collaborative Interactions?. In Proceedings of the 9th International Conference on Computer Supported Collaborative Learning - Volume 1 (Rhodes, Greece) (CSCL'09). International Society of the Learning Sciences, 335--344.
[6]
Eva Hornecker. 2008. "I don't understand it either, but it is cool"? - visitor interactions with a multi-touch table in a museum. 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems (2008), 113--120.
[7]
X. Huang, C. Shen, X. Boix, and Q. Zhao. 2015. SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks. In 2015 IEEE International Conference on Computer Vision (ICCV). 262--270.
[8]
P. Isenberg, D. Fisher, S. A. Paul, M. R. Morris, K. Inkpen, and M. Czerwinski. 2012. Co-Located Collaborative Visual Analytics around a Tabletop Display. IEEE Transactions on Visualization and Computer Graphics, Vol. 18, 5 (May 2012), 689--702. https://doi.org/10.1109/TVCG.2011.287
[9]
Moritz Kassner, William Patera, and Andreas Bulling. 2014. Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction. In Adjunct Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Seattle, Washington) (UbiComp '14 Adjunct). ACM, New York, NY, USA, 1151--1160. https://doi.org/10.1145/2638728.2641695
[10]
Andrew L Kun, Oskar Palinko, Zeljko Medenica, and Peter A Heeman. 2013. On the feasibility of using pupil diameter to estimate cognitive load changes for in-vehicle spoken dialogues. In INTERSPEECH. 3766--3770.
[11]
Q. Lai, W. Wang, H. Sun, and J. Shen. 2020. Video Saliency Prediction Using Spatiotemporal Residual Attentive Networks. IEEE Transactions on Image Processing, Vol. 29 (2020), 1113--1126.
[12]
Hang Li, Julien Epps, and Siyuan Chen. 2020. Think before you speak: An investigation of eye activity patterns during conversations using eyewear. International Journal of Human-Computer Studies (2020), 102468.
[13]
P. Marshall, E. Hornecker, R. Morris, N. S. Dalton, and Y. Rogers. 2008. When the fingers do the talking: A study of group participation with varying constraints to a tabletop interface. In 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems. 33--40.
[14]
Sami Pietinen, Roman Bednarik, and Markku Tukiainen. 2010. Shared Visual Attention in Collaborative Programming: A Descriptive Analysis. In Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering (Cape Town, South Africa) (CHASE '10). ACM, New York, NY, USA, 21--24. https://doi.org/10.1145/1833310.1833314
[15]
Daniel C. Richardson and Rick Dale. 2005. Looking To Understand: The Coupling Between Speakers' and Listeners' Eye Movements and Its Relationship to Discourse Comprehension. Cognitive Science, Vol. 29, 6 (2005), 1045--1060. https://doi.org/10.1207/s15516709cog0000_29
[16]
Yusuke Sugano, Xucong Zhang, and Andreas Bulling. 2016. AggreGaze: Collective Estimation of Audience Attention on Public Displays. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST '16). Association for Computing Machinery, New York, NY, USA, 821--831. https://doi.org/10.1145/2984511.2984536
[17]
Rohail Syed, Kevyn Collins-Thompson, Paul N Bennett, Mengqiu Teng, Shane Williams, Dr Wendy W Tay, and Shamsi Iqbal. 2020. Improving Learning Outcomes with Gaze Tracking and Automatic Question Generation. In Proceedings of The Web Conference 2020. 1693--1703.
[18]
Hidde van der Meulen, Petra Varsanyi, Lauren Westendorf, Andrew L. Kun, and Orit Shaer. 2016. Towards Understanding Collaboration Around Interactive Surfaces: Exploring Joint Visual Attention. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST '16 Adjunct). ACM, New York, NY, USA, 219--220. https://doi.org/10.1145/2984751.2984778
[19]
Robert Walter, Andreas Bulling, David Lindlbauer, Martin Schuessler, and Jorg Muller. 2015. Analyzing Visual Attention During Whole Body Interaction with Public Displays. https://doi.org/10.1145/2750858.2804255
Index Terms
- Towards Computational Identification of Visual Attention on Interactive Tabletops
Recommendations
Video game design using an eye-movement-dependent model of visual attention
Eye movements can be used to infer the allocation of covert attention. In this article, we propose to model the allocation of attention in a task-dependent manner based on different eye movement conditions, specifically fixation and pursuit. We show ...
Human Visual Scanpath Prediction Based on RGB-D Saliency
ICIGP '18: Proceedings of the 2018 International Conference on Image and Graphics ProcessingHuman visual perception is considered as a dynamic process of information acquisition, while the visual scanpath can clearly reflect the shift of our eye fixations. In the previous study of visual attention, researchers generally do the saliency ...
How Shared Visual Attention Patterns of Pairs Unfold Over Time when Workload Changes
ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and ApplicationsData rich environments rely on operator collaboration to manage workload changes. This work explores the relationship between operators’ visual attention and collaborative performance during these workload changes. Percent gaze overlap and percent ...
Comments
Information & Contributors
Information
Published In
November 2020
127 pages
ISBN:9781450375269
DOI:10.1145/3380867
- General Chairs:
- Joaquim Jorge,
- Alfredo Ferreira,
- Program Chairs:
- Nicolai Marquardt,
- Fanny Chevalier
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 November 2020
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
ISS '20
Sponsor:
Acceptance Rates
Overall Acceptance Rate 147 of 533 submissions, 28%
Upcoming Conference
ISS '24
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 86Total Downloads
- Downloads (Last 12 months)10
- Downloads (Last 6 weeks)1
Reflects downloads up to 21 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in