research-article

Just gaze and wave: exploring the use of gaze and gestures for shoulder-surfing resilient authentication

Authors:

Yasmeen Abdrabou,

Mohamed Khamis,

Rana Mohamed Eisa,

Sherif Ismail, and

Amrl ElmougyAuthors Info & Claims

ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

June 2019

Article No.: 29, Pages 1 - 10

https://doi.org/10.1145/3314111.3319837

Published: 25 June 2019 Publication History

Abstract

Eye-gaze and mid-air gestures are promising for resisting various types of side-channel attacks during authentication. However, to date, a comparison of the different authentication modalities is missing. We investigate multiple authentication mechanisms that leverage gestures, eye gaze, and a multimodal combination of them and study their resilience to shoulder surfing. To this end, we report on our implementation of three schemes and results from usability and security evaluations where we also experimented with fixed and randomized layouts. We found that the gaze-based approach outperforms the other schemes in terms of input time, error rate, perceived workload, and resistance to observation attacks, and that randomizing the layout does not improve observation resistance enough to warrant the reduced usability. Our work further underlines the significance of replicating previous eye tracking studies using today's sensors as we show significant improvement over similar previously introduced gaze-based authentication systems.

References

[1]

Yomna Abdelrahman, Mohamed Khamis, Stefan Schneegass, and Florian Alt. 2017. Stay Cool! Understanding Thermal Attacks on Mobile-based User Authentication. In Proceedings of the 35th Annual ACM Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA.

Digital Library

[2]

Yasmeen Abdrabou, Mohamed Khamis, Rana Mohamed Eisa, Sherif Ismael, and Amr Elmougy. 2018. eNGAGE: Resisting Shoulder Surfing Using Novel Gaze Gestures Authentication. In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia (MUM 2018). ACM, New York, NY, USA, 469--473.

Digital Library

[3]

Ilhan Aslan, Andreas Uhl, Alexander Meschtscherjakov, and Manfred Tscheligi. 2014. Mid-air Authentication Gestures: An Exploration of Authentication Based on Palm and Finger Motions. In Proceedings of the 16th International Conference on Multimodal Interaction (ICMI '14). ACM, New York, NY, USA, 311--318.

Digital Library

[4]

Ilhan Aslan, Andreas Uhl, Alexander Meschtscherjakov, and Manfred Tscheligi. 2016. Design and Exploration of Mid-Air Authentication Gestures. ACM Trans. Interact. Intell. Syst. 6, 3, Article 23 (Sept. 2016), 22 pages.

Digital Library

[5]

Adam J. Aviv, Katherine Gibson, Evan Mossop, Matt Blaze, and Jonathan M. Smith. 2010. Smudge Attacks on Smartphone Touch Screens. In Proceedings of the 4th USENIX Conference on Offensive Technologies (WOOT'10). USENIX Association, Berkeley, CA, USA, 1--7. http://dl.acm.org/citation.cfm?id=1925004.1925009

Digital Library

[6]

Andrea Bianchi. 2011. Authentication on Public Terminals with Private Devices. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11). ACM, New York, NY, USA, 429--430.

Digital Library

[7]

Andrea Bianchi, Ian Oakley, Vassilis Kostakos, and Dong Soo Kwon. 2011. The Phone Lock: Audio and Haptic Shoulder-surfing Resistant PIN Entry Methods for Mobile Devices. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11). ACM, New York, NY, USA, 197--200.

Digital Library

[8]

Andrea Bianchi, Ian Oakley, and Dong Soo Kwon. 2012. Counting Clicks and Beeps: Exploring Numerosity Based Haptic and Audio PIN Entry. Interact. Comput. 24, 5 (Sept. 2012), 409--422.

Digital Library

[9]

Alexander De Luca, Martin Denzel, and Heinrich Hussmann. 2009. Look into My Eyes!: Can You Guess My Password?. In Proceedings of the 5th Symposium on Usable Privacy and Security (SOUPS '09). ACM, New York, NY, USA, Article 7, 12 pages.

Digital Library

[10]

Alexander De Luca, Alina Hang, Frederik Brudy, Christian Lindner, and Heinrich Hussmann. 2012. Touch Me Once and I Know It's You!: Implicit Authentication Based on Touch Screen Patterns. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 987--996.

Digital Library

[11]

Alexander De Luca, Alina Hang, Emanuel von Zezschwitz, and Heinrich Hussmann. 2015. I Feel Like I'M Taking Selfies All Day!: Towards Understanding Biometric Authentication on Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1411--1414.

Digital Library

[12]

Alexander De Luca, Marian Harbach, Emanuel von Zezschwitz, Max-Emanuel Maurer, Bernhard Ewald Slawik, Heinrich Hussmann, and Matthew Smith. 2014. Now You See Me, Now You Don't: Protecting Smartphone Authentication from Shoulder Surfers. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2937--2946.

Digital Library

[13]

Alexander De Luca, Emanuel von Zezschwitz, Laurent Pichler, and Heinrich Hussmann. 2013. Using Fake Cursors to Secure On-screen Password Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2399--2402.

Digital Library

[14]

Alexander De Luca, Roman Weiss, and Heiko Drewes. 2007. Evaluation of Eye-gaze Interaction Methods for Security Enhanced PIN-entry. In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces (OZCHI '07). ACM, New York, NY, USA, 199--202.

Digital Library

[15]

Heiko Drewes, Alexander De Luca, and Albrecht Schmidt. 2007. Eye-gaze Interaction for Mobile Phones. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology (Mobility '07). ACM, New York, NY, USA, 364--371.

Digital Library

[16]

Malin Eiband, Mohamed Khamis, Emanuel von Zezschwitz, Heinrich Hussmann, and Florian Alt. 2017. Understanding Shoulder Surfing in the Wild: Stories from Users and Observers. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 11.

Digital Library

[17]

Alain Forget, Sonia Chiasson, and Robert Biddle. 2010. Shoulder-surfing Resistance with Eye-gaze Entry in Cued-recall Graphical Passwords. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 1107--1110.

Digital Library

[18]

Ceenu Goerge, Mohamed Khamis, Emanuel von Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017. Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality. In Proceedings of the Network and Distributed System Security Symposium (USEC '17). NDSS.

[19]

Eiji Hayashi, Manuel Maas, and Jason I. Hong. 2014. Wave to Me: User Identification Using Body Lengths and Natural Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3453--3462.

Digital Library

[20]

Ponemon Institute. 2016. Global Visual Hacking Experimental Study: Analysis. multimedia.3m.com/mws/media/1254232O/global-visual-hacking-experiment-study-summary.pdf

[21]

Mohamed Khamis, Florian Alt, Mariam Hassib, Emanuel von Zezschwitz, Regina Hasholzner, and Andreas Bulling. 2016. GazeTouchPass: Multimodal Authentication Using Gaze and Touch on Mobile Devices. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 2156--2164.

Digital Library

[22]

Mohamed Khamis, Linda Bandelow, Stina Schick, Dario Casadevall, Andreas Bulling, and Florian Alt. 2017a. They Are All After You: Investigating the Viability of a Threat Model That Involves Multiple Shoulder Surfers. In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia (MUM '17). ACM, New York, NY, USA, 31--35.

Digital Library

[23]

Mohamed Khamis, Regina Hasholzner, Andreas Bulling, and Florian Alt. 2017b. GTmoPass: Two-factor Authentication on Public Displays Using GazeTouch passwords and Personal Mobile Devices. In Proceedings of the 6th International Symposium on Pervasive Displays (PerDis '17). ACM, New York, NY, USA, 9.

Digital Library

[24]

Mohamed Khamis, Mariam Hassib, Emanuel von Zezschwitz, Andreas Bulling, and Florian Alt. 2017c. GazeTouchPIN: Protecting Sensitive Data on Mobile Devices Using Secure Multimodal Authentication. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI 2017). ACM, New York, NY, USA, 446--450.

Digital Library

[25]

Mohamed Khamis, Ludwig Trotter, Ville Mäkelä, Emanuel von Zezschwitz, Jens Le, Andreas Bulling, and Florian Alt. 2018. CueAuth: Comparing Touch, Mid-Air Gestures, and Gaze for Cue-based Authentication on Situated Displays. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 174 (Dec. 2018), 21 pages.

Digital Library

[26]

Manu Kumar, Tal Garfinkel, Dan Boneh, and Terry Winograd. 2007. Reducing Shoulder-surfing by Using Gaze-based Password Entry. In Proceedings of the 3rd Symposium on Usable Privacy and Security (SOUPS '07). ACM, New York, NY, USA, 13--19.

Digital Library

[27]

Keaton Mowery, Sarah Meiklejohn, and Stefan Savage. 2011. Heat of the Moment: Characterizing the Efficacy of Thermal Camera-based Attacks. In Proceedings of the 5th USENIX Conference on Offensive Technologies (WOOT'11). USENIX Association, Berkeley, CA, USA, 6--6. http://dl.acm.org/citation.cfm?id=2028052.2028058

Digital Library

[28]

Takashi Nagamatsu, Junzo Kamahara, Takumi Iko, and Naoki Tanaka. 2008. One-point Calibration Gaze Tracking Based on Eyeball Kinematics Using Stereo Cameras. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (ETRA '08). ACM, New York, NY, USA, 95--98.

Digital Library

[29]

Florian Schaub, Marcel Walch, Bastian Könings, and Michael Weber. 2013. Exploring the Design Space of Graphical Passwords on Smartphones. In Proceedings of the Ninth Symposium on Usable Privacy and Security (SOUPS '13). ACM, New York, NY, USA, Article 11, 14 pages.

Digital Library

[30]

Ivo Sluganovic, Marc Roeschlin, Kasper B. Rasmussen, and Ivan Martinovic. 2016. Using Reflexive Eye Movements for Fast Challenge-Response Authentication. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16). ACM, New York, NY, USA, 1056--1067.

Digital Library

[31]

Chen Song, Aosen Wang, Kui Ren, and Wenyao Xu. 2016. "EyeVeri: A Secure and Usable Approach for Smartphone User Authentication". In IEEE International Conference on Computer Communication (INFOCOM'16). San Francisco, California, 1 -- 9.

Digital Library

[32]

Martin Stokkenes, Raghavendra Ramachandra, and Christoph Busch. 2016. Biometric Authentication Protocols on Smartphones: An Overview. In Proceedings of the 9th International Conference on Security of Information and Networks (SIN '16). ACM, New York, NY, USA, 136--140.

Digital Library

[33]

Furkan Tari, A. Ant Ozok, and Stephen H. Holden. 2006. A Comparison of Perceived and Real Shoulder-surfing Risks Between Alphanumeric and Graphical Passwords. In Proceedings of the Second Symposium on Usable Privacy and Security (SOUPS '06). ACM, New York, NY, USA, 56--66.

Digital Library

[34]

Emanuel von Zezschwitz, Alexander De Luca, Bruno Brunkow, and Heinrich Hussmann. 2015a. SwiPIN: Fast and Secure PIN-Entry on Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1403--1406.

Digital Library

[35]

Emanuel von Zezschwitz, Alexander De Luca, Philipp Janssen, and Heinrich Hussmann. 2015b. Easy to Draw, but Hard to Trace?: On the Observability of Grid-based (Un)Lock Patterns. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2339--2342.

Digital Library

[36]

Emanuel von Zezschwitz, Paul Dunphy, and Alexander De Luca. 2013. Patterns in the Wild: A Field Study of the Usability of Pattern and Pin-based Authentication on Mobile Devices. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '13). ACM, New York, NY, USA, 261--270.

Digital Library

[37]

Oliver Wiese and Volker Roth. 2016. See You Next Time: A Model for Modern Shoulder Surfers. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 453--464.

Digital Library

[38]

Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2015b. Appearance-Based Gaze Estimation in the Wild. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]

Yulong Zhang, Zhaonfeng Chen, Hui Xue, and Tao Wei. 2015a. Fingerprints On Mobile Devices: Abusing and leaking. In Black Hat Conference.

Cited By

Cilia NArena ABarbera ABorrello ECapizzi GCappello SFaletra LIncardone SSorce S(2024)A pilot study on gaze and mouse data for user identificationProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655647(1-3)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655647
Abdrabou YOmelina TDietz FKhamis MAlt FHassib M(2024)Where Do You Look When Unlocking Your Phone? : A Field Study of Gaze Behaviour During Smartphone UnlockExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651094(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651094
Price ALoizides F(2024)Allowing for Secure and Accessible Authentication for Individuals with Disabilities of DexterityHuman-Centered Software Engineering10.1007/978-3-031-64576-1_7(133-146)Online publication date: 1-Jul-2024
https://doi.org/10.1007/978-3-031-64576-1_7
Show More Cited By

Index Terms

Just gaze and wave: exploring the use of gaze and gestures for shoulder-surfing resilient authentication
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Security and privacy

Recommendations

Hide my Gaze with EOG!: Towards Closed-Eye Gaze Gesture Passwords that Resist Observation-Attacks with Electrooculography in Smart Glasses
MoMM2019: Proceedings of the 17th International Conference on Advances in Mobile Computing & Multimedia

Smart glasses allow for gaze gesture passwords as a hands-free form of mobile authentication. However, pupil movements for password input are easily observed by attackers, who thereby can derive the password. In this paper we investigate closed-eye gaze ...
Read More
Gaze-Hand Alignment: Combining Eye Gaze and Mid-Air Pointing for Interacting with Menus in Augmented Reality
ETRA

Gaze and freehand gestures suit Augmented Reality as users can interact with objects at a distance without need for a separate input device. We propose Gaze-Hand Alignment as a novel multimodal selection principle, defined by concurrent use of both gaze ...
Read More
Design and Exploration of Mid-Air Authentication Gestures
Regular Articles and Special Issue on Highlights of ICMI 2014 (Part 2 of 2)

Authentication based on touchless mid-air gestures would benefit a multitude of ubiquitous computing applications, especially those that are used in clean environments (e.g., medical environments or clean rooms). In order to explore the potential of mid-...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

June 2019

623 pages

ISBN:9781450367097

DOI:10.1145/3314111

Conference Chairs:
Krzysztof Krejtz
SWPS University, Poland
,
Bonita Sharif
University of Nebraska-Lincoln

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ETRA '19

Sponsor:

ETRA '19: 2019 Symposium on Eye Tracking Research and Applications

June 25 - 28, 2019

Colorado, Denver

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
469
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)6

Other Metrics

View Author Metrics

Citations

Cited By

Cilia NArena ABarbera ABorrello ECapizzi GCappello SFaletra LIncardone SSorce S(2024)A pilot study on gaze and mouse data for user identificationProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655647(1-3)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655647
Abdrabou YOmelina TDietz FKhamis MAlt FHassib M(2024)Where Do You Look When Unlocking Your Phone? : A Field Study of Gaze Behaviour During Smartphone UnlockExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651094(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651094
Price ALoizides F(2024)Allowing for Secure and Accessible Authentication for Individuals with Disabilities of DexterityHuman-Centered Software Engineering10.1007/978-3-031-64576-1_7(133-146)Online publication date: 1-Jul-2024
https://doi.org/10.1007/978-3-031-64576-1_7
Marky KMacdonald SAbdrabou YKhamis MCalandrino JTroncoso C(2023)In the quest to protect users from side-channel attacksProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620530(5235-5252)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620530
Namnakani OAbdrabou YGrizou JEsteves AKhamis M(2023)Comparing Dwell time, Pursuits and Gaze Gestures for Gaze Interaction on Handheld Mobile DevicesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580871(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580871
Zhou LWang KLai JZhang D(2023)A Comparison of a Touch-Gesture- and a Keystroke-Based Password Method: Toward Shoulder-Surfing Resistant Mobile User AuthenticationIEEE Transactions on Human-Machine Systems10.1109/THMS.2023.323632853:2(303-314)Online publication date: Apr-2023
https://doi.org/10.1109/THMS.2023.3236328
Wang KZhou LZhang DLai J(2023)Shoulder Surfing on Mobile Authentication: Perception vis-a-vis Performance from the Attacker's Perspective2023 IEEE International Conference on Intelligence and Security Informatics (ISI)10.1109/ISI58743.2023.10297219(1-6)Online publication date: 2-Oct-2023
https://doi.org/10.1109/ISI58743.2023.10297219
Kirkwood DTombul CFirth CMacdonald FPriftis KMathis FKhamis MMarky K(2022)PIN Scrambler: Assessing the Impact of Randomized Layouts on the Usability and Security of PINsProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia10.1145/3568444.3568450(83-88)Online publication date: 27-Nov-2022
https://dl.acm.org/doi/10.1145/3568444.3568450
Kamarushi MWatson STigwell GPeiris R(2022)OneButtonPIN: A Single Button Authentication Method for Blind or Low Vision Users to Improve Accessibility and Prevent EavesdroppingProceedings of the ACM on Human-Computer Interaction10.1145/35467476:MHCI(1-22)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1145/3546747
Mathis FVaniea KKhamis M(2022)Can I Borrow Your ATM? Using Virtual Reality for (Simulated) In Situ Authentication Research2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR51125.2022.00049(301-310)Online publication date: Mar-2022
https://doi.org/10.1109/VR51125.2022.00049
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents