research-article

Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis

Authors:

Jianbing Xiahou,

Yingxuan JiAuthors Info & Claims

CGI 2018: Proceedings of Computer Graphics International 2018

Pages 185 - 194

https://doi.org/10.1145/3208159.3208180

Published: 11 June 2018 Publication History

Abstract

The human eye's state of motion and content of interest can express people's cognitive status and emotional status based on their situation. When observing the surrounding things, the human eyes make different eye movements according to the observed objects which reflects human's attention and interest. In this paper, we capture and analyze patterns of human eye-gaze behavior and head motion and classify them into different categories. Besides, we compute and train the eye-object movement attention model and eye-object feature preference model based on different peoples' eye-gaze behaviors by using machine learning algorithms. These models are used to predict humans' object of interest and the interaction intention according to people's real-time situation. Furthermore, the eye-gaze behavior and head motion patterns can be used as a modality of non-verbal information in the computing of human emotional states based on the PAD affective computing model. Our methodology analyzes human emotion and cognition status from the aspect of eye-gaze behavior and head motion, understands the cognitive information that human eyes can express, and effectively improves the efficiency of human-computer interaction in different circumstances.

References

[1]

Langton S R H. How the eyes affect the I: gaze perception, cognition, and the robot-human interface." IEEE International Workshop on Robot and Human Interactive Communication, 2001. Proceedings. IEEE, 2001:359--365.

[2]

Maria Guarnera, Zira Hichy, Maura I. Cascio, Stefano Carrubba "Facial Expressions and Ability to Recognize Emotions From Eyes or Mouth in Children", Europe's Journal of Psychology, Vol. 11(2), 2015

[3]

Lance B, Marsella S C. The Relation between Gaze Behavior and the Attribution of Emotion: An Empirical Study{M}// Intelligent Virtual Agents. Springer Berlin Heidelberg, 2008:1--14.

Digital Library

[4]

https://www.tobii.com/group/news-media/press-releases/2017/8/tobii-and-microsoft-collaborate-to-bring-eye-tracking-support-in-windows-10/

[5]

Vicente F, Huang Z, Xiong X, et al. Driver Gaze Tracking and Eyes Off the Road Detection System{J}. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(4):2014--2027.

[6]

Sundstedt V. Gazing at games:using eye tracking to control virtual characters." 2010:1--160.

Digital Library

[7]

Borys M, Barakate S, Hachmoud K, et al. Classification of user performance in the Ruff Figural Fluency Test based on eye-tracking features{J}. 2017, 15(380):02002.

[8]

Reading performance Using Eye Tracking to Assess Reading performance in Patients with Glaucoma: A Within-Person Study, Nicholas D. Smith, Fiona C. Glen, Vera M. Mönter, and David P. Crabb, Hindawi Publishing Corporation Journal of Ophthalmology Volume 2014, Article ID 120528, 10 pages

[9]

Hofler M, Wesiak G, Purcher P, et al. Modern education and its background in cognitive psychology: Automated question creation and eye movements." International Convention on Information and Communication Technology, Electronics and Microelectronics. 2017:619--623.

[10]

Swearingen T, Ross A. A Label Propagation Approach for Predicting Missing Biographic Labels in Face-Based Biometric Records{J}. 2017.

[11]

Li Q, Li P, Zhang Q, et al. Icing load prediction for overhead power lines based on SVM." International Conference on Modelling, Identification and Control. IEEE, 2011:104--108.

[12]

Buschjäger S, Morik K. Decision Tree and Random Forest Implementations for Fast Filtering of Sensor Data{J}. IEEE Transactions on Circuits & Systems I Regular Papers, 2017, PP(99):1--14.

[13]

https://en.wikipedia.org/wiki/Kinematics

[14]

Wade J, Zhang L, Bian D, et al. A Gaze-Contingent Adaptive Virtual Reality Driving Environment for Intervention in Individuals with Autism Spectrum Disorders{J}. Acm Transactions on Interactive Intelligent Systems, 2016, 6(1):3.

Digital Library

[15]

Ruhland K, Peters C E, Andrist S, et al. A Review of Eye Gaze in Virtual Agents, Social Robotics and HCI: Behaviour Generation, User Interaction and Perception{J}. Computer Graphics Forum, 2015, 34(6):299--326.

Digital Library

[16]

Bâce M. Augmenting human interaction capabilities with proximity, natural gestures, and eye gaze." The, International Conference. 2017:1--3.

Digital Library

[17]

Zhao Q, Yuan X, Tu D, et al. Eye moving behaviors identification for gaze tracking interaction{J}. Journal on Multimodal User Interfaces, 2015, 9(2):89--104.

[18]

Zheng R, Nakano K, Ishiko H, et al. Eye-Gaze Tracking Analysis of Driver Behavior While Interacting With Navigation Systems in an Urban Area{J}. IEEE Transactions on Human-Machine Systems, 2016, 46(4):546--556.

[19]

Kern D, Marshall P, Schmidt A. Gazemarks:gaze-based visual placeholders to ease attention switching." Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2010:2093--2102.

Digital Library

[20]

Yun S P, Outram B I, Tag B, et al. GazeSphere: navigating 360-degree-video environments in VR using head rotation and eye gaze." ACM SIGGRAPH. ACM, 2017:1--2.

Digital Library

[21]

Springer. Journal on Multimodal User Interfaces{J}. 2015.

[22]

Xiahou J, He H, Wei K, et al. Integrated Approach of Dynamic Human Eye Movement Recognition and Tracking in Real-time." International Conference on Virtual Reality and Visualization. IEEE, 2017:94--101.

[23]

Kao T C, Sun T Y. Head pose recognition in advanced Driver Assistance System." IEEE, Global Conference on Consumer Electronics. IEEE, 2017:1--3.

[24]

C. Z. Liu, H. Aliamani and M. Kavakli, "Behavior-intention analysis and human-aware computing: Case study and discussion," 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA), Siem Reap, Cambodia, 2017, pp. 516--521.

[25]

Albert Mehrabian. Pleasure-arousaldominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4):261--292, 1996.

[26]

Brent Lance and Stacy C Marsella. The relation between gaze behavior and the attribution of emotion: An empirical study. In International Workshop on Intelligent Virtual Agents, pages 1--14. Springer, 2008.

Digital Library

[27]

Albert Mehrabian. Pleasure-arousaldominance: A general framework for describing and measuring individual differences in temperament. Current Psychology,14(4):261--292, 1996.

Cited By

Goyal SLaddi A(2024)Machine Learning Techniques for Emotion Detection Using Eye Gaze LocalisationMachine and Deep Learning Techniques for Emotion Detection10.4018/979-8-3693-4143-8.ch002(24-60)Online publication date: 14-May-2024
https://doi.org/10.4018/979-8-3693-4143-8.ch002
Mamykin GKulesh ABarkov FKonstantinov YSokol’chik DPervadchuk V(2024)Methods for Detecting the Patient’s Pupils’ Coordinates and Head Rotation Angle for the Video Head Impulse Test (vHIT), Applicable for the Diagnosis of Vestibular Neuritis and Pre-Stroke ConditionsComputation10.3390/computation1208016712:8(167)Online publication date: 18-Aug-2024
https://doi.org/10.3390/computation12080167
Kim SLee SLee E(2024)Advancements in Gaze Coordinate Prediction Using Deep Learning: A Novel Ensemble Loss ApproachApplied Sciences10.3390/app1412533414:12(5334)Online publication date: 20-Jun-2024
https://doi.org/10.3390/app14125334
Show More Cited By

Index Terms

Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Eye-gaze interaction for mobile phones
Mobility '07: Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology

In this paper, we discuss the use of eye-gaze tracking technology for mobile phones. In particular we investigate how gaze interaction can be used to control applications on handheld devices. In contrast to eye-tracking systems for desktop computers, ...
An eye-gaze oriented context based interaction paradigm design
HCI '18: Proceedings of the 32nd International BCS Human Computer Interaction Conference

The human eye's state of motion and content of interest can express people's cognitive status based on their situations. When observing the surroundings, the human eyes make different eye movements to interact with the observed objects which reflects ...
Augmenting human interaction capabilities with proximity, natural gestures, and eye gaze
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

Nowadays, humans are surrounded by many complex computer systems. When people interact among each other, they use multiple modalities including voice, body posture, hand gestures, facial expressions, or eye gaze. Currently, computers can only understand ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CGI 2018: Proceedings of Computer Graphics International 2018

June 2018

284 pages

ISBN:9781450364010

DOI:10.1145/3208159

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CGI 2018

CGI 2018: Computer Graphics International 2018

June 11 - 14, 2018

Island, Bintan, Indonesia

Acceptance Rates

CGI 2018 Paper Acceptance Rate 35 of 159 submissions, 22%;

Overall Acceptance Rate 35 of 159 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
714
Total Downloads

Downloads (Last 12 months)124
Downloads (Last 6 weeks)14

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Goyal SLaddi A(2024)Machine Learning Techniques for Emotion Detection Using Eye Gaze LocalisationMachine and Deep Learning Techniques for Emotion Detection10.4018/979-8-3693-4143-8.ch002(24-60)Online publication date: 14-May-2024
https://doi.org/10.4018/979-8-3693-4143-8.ch002
Mamykin GKulesh ABarkov FKonstantinov YSokol’chik DPervadchuk V(2024)Methods for Detecting the Patient’s Pupils’ Coordinates and Head Rotation Angle for the Video Head Impulse Test (vHIT), Applicable for the Diagnosis of Vestibular Neuritis and Pre-Stroke ConditionsComputation10.3390/computation1208016712:8(167)Online publication date: 18-Aug-2024
https://doi.org/10.3390/computation12080167
Kim SLee SLee E(2024)Advancements in Gaze Coordinate Prediction Using Deep Learning: A Novel Ensemble Loss ApproachApplied Sciences10.3390/app1412533414:12(5334)Online publication date: 20-Jun-2024
https://doi.org/10.3390/app14125334
Hassan AAbdelghany RHegazy OAshraf YAtia A(2024)Embedding Eye Gaze and Emotions into Tangible Social Media for Transforming Vandalism into Arts2024 International Telecommunications Conference (ITC-Egypt)10.1109/ITC-Egypt61547.2024.10620597(836-841)Online publication date: 22-Jul-2024
https://doi.org/10.1109/ITC-Egypt61547.2024.10620597
da Silveira ALima de Souza MGhinea GSaibel Santos C(2024)Physiological Data for User Experience and Quality of Experience: A Systematic Review (2018–2022)International Journal of Human–Computer Interaction10.1080/10447318.2024.2311972(1-30)Online publication date: 13-Feb-2024
https://doi.org/10.1080/10447318.2024.2311972
Li YChen JMa JWang XZhang W(2023)Gaze Estimation Based on Convolutional Structure and Sliding Window-Based Attention MechanismSensors10.3390/s2313622623:13(6226)Online publication date: 7-Jul-2023
https://doi.org/10.3390/s23136226
Ghasemi YJeong HPark KChoi SLee J(2023)Evaluating User Interactions in Wearable Extended Reality: Modeling, Online Remote Survey, and In-Lab Experimental MethodsIEEE Access10.1109/ACCESS.2023.329859811(77856-77872)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3298598
Bature ZAbdullahi SYeamkuan SChiracharit WChamnongthai K(2023)Boosted Gaze Gesture Recognition Using Underlying Head Orientation SequenceIEEE Access10.1109/ACCESS.2023.327028511(43675-43689)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3270285
Akshay SKavya Bijith PSanjana SAmudha J(2023)iBEHAVE: Behaviour Analysis Using Eye Gaze MetricesPattern Recognition and Machine Intelligence10.1007/978-3-031-45170-6_27(260-269)Online publication date: 4-Dec-2023
https://doi.org/10.1007/978-3-031-45170-6_27
Ghasemi YJeong H(2022)Using Gaze-based Interaction to Alleviate Situational Mobility Impairment in Extended RealityProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132266122466:1(435-439)Online publication date: 27-Oct-2022
https://doi.org/10.1177/1071181322661224
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents