research-article

Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality

Authors:

Matthias J. Geiger,

Daniel Buschek,

Florian AltAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 110, Pages 1 - 12

https://doi.org/10.1145/3290605.3300340

Published: 02 May 2019 Publication History

Abstract

Every person is unique, with individual behavioural characteristics: how one moves, coordinates, and uses their body. In this paper we investigate body motion as behavioural biometrics for virtual reality. In particular, we look into which behaviour is suitable to identify a user. This is valuable in situations where multiple people use a virtual reality environment in parallel, for example in the context of authentication or to adapt the VR environment to users' preferences. We present a user study (N=22) where people perform controlled VR tasks (pointing, grabbing, walking, typing), monitoring their head, hand, and eye motion data over two sessions. These body segments can be arbitrarily combined into body relations, and we found that these movements and their combination lead to characteristic behavioural patterns. We present an extensive analysis of which motion/relation is useful to identify users in which tasks using classification methods. Our findings are beneficial for researchers and practitioners alike who aim to build novel adaptive and secure user interfaces in virtual reality.

Supplementary Material

ZIP File (pn7792.zip)

The zip file contains one PDF file. It provides the user identification accuracy and F1 scores. The paper does not report F1 to save space, thus it is here a supplementary material.

Download
68.98 KB

MP4 File (paper110p.mp4)

Preview video

Download
632.94 KB

MP4 File (pn7792.mp4)

Supplemental video

Download
11.38 MB

References

[1]

Virginia O. Andersson and Ricardo M. Araujo. 2015. Person Identification Using Anthropometric and Gait Data from Kinect Sensor. In Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality CHI 2019, May 4--9, 2019, Glasgow, Scotland Uk Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI'15). AAAI Press, 425--431. http://dl.acm.org/citation. cfm?id=2887007.2887067

Digital Library

[2]

Corey Ashby, Amit Bhatia, Francesco Tenore, and Jacob Vogelstein. 2011. Low-cost electroencephalogram (EEG) based authentication. In Neural Engineering (NER), 2011 5th International IEEE/EMBS Conference on. IEEE, 442--445.

[3]

Kyle O Bailey, James S Okolica, and Gilbert L Peterson. 2014. User identification and authentication using multi-modal behavioral biometrics. Computers & Security 43 (2014), 77--89.

[4]

VE Belenkii, VS Gurfinkel, and EI Paltsev. 1967. ON THE CONTROL ELEMENTS OF VOLUNTARY MOVEMENTS. Biofizika (1967).

[5]

Chiraz BenAbdelkader, Ross Cutler, and Larry Davis. 2002. Viewinvariant estimation of height and stride for gait recognition. In International Workshop on Biometric Authentication. Springer, 155--167.

Digital Library

[6]

Mark Billinghurst, Jerry Bowskill, Nick Dyer, and Jason Morphett. 1998. Spatial information displays on a wearable computer. IEEE Computer Graphics and Applications 18, 6 (1998), 24--31.

Digital Library

[7]

Cheng Bo, Lan Zhang, Xiang-Yang Li, Qiuyuan Huang, and Yu Wang. 2013. Silentsense: silent user identification via touch and movement behavioral biometrics. In Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 187--190.

Digital Library

[8]

Edward G Britton, James S Lipscomb, and Michael E Pique. 1978. Making nested rotations convenient for the user. In ACM SIGGRAPH Computer Graphics, Vol. 12. ACM, 222--227.

Digital Library

[9]

Grigore C Burdea and Philippe Coiffet. 2003. Virtual reality technology. John Wiley & Sons.

Digital Library

[10]

Daniel Buschek, Alexander De Luca, and Florian Alt. 2015. Improving Accuracy, Applicability and Usability of Keystroke Biometrics on Mobile Touchscreen Devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1393--1402.

Digital Library

[11]

Antitza Dantcheva, Carmelo Velardo, Angela D'angelo, and JeanLuc Dugelay. 2011. Bag of soft biometrics for person identification. Multimedia Tools and Applications 51, 2 (2011), 739--777.

Digital Library

[12]

Alexander De Luca, Alina Hang, Frederik Brudy, Christian Lindner, and Heinrich Hussmann. 2012. Touch Me Once and I Know It's You!: Implicit Authentication Based on Touch Screen Patterns. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 987--996.

Digital Library

[13]

Simon Eberz, Kasper B Rasmussen, Vincent Lenders, and Ivan Martinovic. 2017. Evaluating behavioral biometrics for continuous authentication: Challenges and metrics. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. ACM, 386--399.

Digital Library

[14]

Shaun Gallagher and Daniel Schmicking. 2010. Handbook of phenomenology and cognitive science. Springer.

[15]

Yves Guiard. 1987. Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. In Journal of Motor Behavior, Vol. 19. 486--517.

[16]

Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3011--3014.

Digital Library

[17]

ISO ISO. {n. d.}. 9241--9 Ergonomic requirements for office work with visual display terminals (VDTs)-Part 9: Requirements for non-keyboard input devices, 2000. International Organization for Standardization ({n. d.}).

[18]

Robert J.K. Jacob, Audrey Girouard, Leanne M. Hirshfield, Michael S. Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum. 2008. Reality-based Interaction: A Framework for post-WIMP Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 201--210.

Digital Library

[19]

Anil K Jain, Sarat C Dass, and Karthik Nandakumar. 2004. Soft biometric traits for personal recognition systems. In Biometric authentication. Springer, 731--738.

[20]

Erno Jeges, Istvan Kispal, and Zoltan Hornak. 2008. Measuring human height using calibrated cameras. In Human System Interactions, 2008 Conference on. IEEE, 755--760.

[21]

Roland S Johansson, Göran Westling, Anders Bäckström, and J Randall Flanagan. 2001. Eye--hand coordination in object manipulation. Journal of Neuroscience 21, 17 (2001), 6917--6932.

[22]

Georgios Kambourakis, Dimitrios Damopoulos, Dimitrios Papamartzivanos, and Emmanouil Pavlidakis. 2016. Introducing touchstroke: keystroke-based authentication system for smartphones. Security and Communication Networks 9, 6 (2016), 542--554.

Digital Library

[23]

George H. Kohavi, Ron and John. 1997. Wrappers for feature subset selection. Artificial Intelligence 97, 1 (1997), 273--324.

Digital Library

[24]

Alexander Kupin, Benjamin Moeller, Yijun Jiang, Natasha Kholgade Banerjee, and Sean Banerjee. 2019. Task-Driven Biometric Authentication of Users in Virtual Reality (VR) Environments. In MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8--11, 2019, Proceedings, Part I. 55--67.

[25]

Nurakhmed Nurislamovich Latypov and Nurulla Nurislamovich Latypov. 1999. Method for tracking and displaying user's spatial position and orientation, a method for representing virtual reality for a user, and systems of embodiment of such methods. US Patent 6,005,548.

[26]

Sugang Li, Ashwin Ashok, Yanyong Zhang, Chenren Xu, Janne Lindqvist, and Macro Gruteser. 2016. Whose move is it anyway? Authenticating smart wearable devices using unique head movement patterns. In Pervasive Computing and Communications (PerCom), 2016 IEEE International Conference on. IEEE, 1--9.

[27]

Hanneke Liesker, Eli Brenner, and Jeroen BJ Smeets. 2009. Combining eye and hand in search is suboptimal. Experimental brain research 197, 4 (2009), 395--401.

[28]

Feng Lin, Kun Woo Cho, Chen Song, Wenyao Xu, and Zhanpeng Jin. 2018. Brain Password: A Secure and Truly Cancelable Brain Biometrics for Smart Headwear. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 296-- 309.

Digital Library

[29]

Pedro Lopes, Alexandra Ion, Willi Müller, Daniel Hoffmann, Patrik Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, New York, NY, USA, 175--175.

Digital Library

[30]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, New York, NY, USA, 754--755.

Digital Library

[31]

Jean Massion. 1992. Movement, posture and equilibrium: interaction and coordination. Progress in neurobiology 38, 1 (1992), 35--56.

[32]

B Miller. 1988. Everything you need to know about biometric identification. Personal Identification News 1988 Biometric Industry Directory (1988).

[33]

Mark R Mine, Frederick P Brooks Jr, and Carlo H Sequin. 1997. Moving objects in space: exploiting proprioception in virtual-environment interaction. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 19--26. CHI 2019, May 4--9, 2019, Glasgow, Scotland Uk Pfeuffer et al.

Digital Library

[34]

Brent C. Munsell, Andrew Temlyakov, Chengzheng Qu, and Song Wang. 2012. Person Identification Using Full-body Motion and Anthropometric Biometrics from Kinect Videos. In Proceedings of the 12th International Conference on Computer Vision - Volume Part III (ECCV'12). Springer-Verlag, Berlin, Heidelberg, 91--100.

Digital Library

[35]

Tahrima Mustafa, Richard Matovu, Abdul Serwadda, and Nicholas Muirhead. 2018. Unsure How to Authenticate on Your VR Headset?: Come on, Use Your Head!. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics (IWSPA '18). ACM, New York, NY, USA, 23--30.

Digital Library

[36]

Vitruvius Pollio and M. H. Morgan. 1960. De architectura. New York: Dover Publications.

[37]

Daniel A Reid and Mark S Nixon. 2011. Using comparative human descriptions for soft biometrics. In Biometrics (IJCB), 2011 International Joint Conference on. IEEE, 1--6.

Digital Library

[38]

Ioannis Rigas, Oleg Komogortsev, and Reza Shadmehr. 2016. Biometric recognition via eye movements: Saccadic vigor and acceleration cues. ACM Transactions on Applied Perception (TAP) 13, 2 (2016), 6.

Digital Library

[39]

Yiran Shen, Hongkai Wen, Chengwen Luo, Weitao Xu, Tao Zhang, Wen Hu, and Daniela Rus. 2018. GaitLock: Protect virtual and augmented reality headsets using gait. IEEE Transactions on Dependable and Secure Computing (2018).

[40]

Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-based Text Entry in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 647, 13 pages.

Digital Library

[41]

Robert John Teather. 2013. Evaluating 3D pointing techniques. (2013).

[42]

Robert J Teather and Wolfgang Stuerzlinger. 2011. Pointing at 3D targets in a stereo head-tracked virtual environment. In 3D User Interfaces (3DUI), 2011 IEEE Symposium on. IEEE, 87--94.

Digital Library

[43]

Colin Ware and Harutune H. Mikaelian. 1987. An Evaluation of an Eye Tracker As a Device for Computer Input. In Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface (CHI '87). ACM, New York, NY, USA, 183--188.

Digital Library

[44]

J Wayman. 2000. A definition of biometrics. National Biometric Test Center Collected Works 1, 2 (2000), 21--23.

[45]

Shanhe Yi, Zhengrui Qin, Ed Novak, Yafeng Yin, and Qun Li. 2016. Glassgesture: Exploring head gesture interface of smart glasses. In Computer Communications, IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on. IEEE, 1--9.

Digital Library

[46]

Yongtuo Zhang, Wen Hu, Weitao Xu, Chun Tung Chou, and Jiankun Hu. 2018. Continuous Authentication Using Eye Movement Response of Implicit Visual Stimuli. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 177.

Digital Library

[47]

Zhaoxiang Zhang, Maodi Hu, and Yunhong Wang. 2011. A survey of advances in biometric gait recognition. In Chinese Conference on Biometric Recognition. Springer, 150--158.

Digital Library

[48]

Zhaoxiang Zhang, Kaiyue Wang, and Yunhong Wang. 2011. A survey of on-line signature verification. In Chinese Conference on Biometric Recognition. Springer, 141--149.

Digital Library

Cited By

Wu YZhang XWu TZhou BNguyen PLiu J(2025)3D Facial Tracking and User Authentication Through Lightweight Single-Ear BiosensorsIEEE Transactions on Mobile Computing10.1109/TMC.2024.347033924:2(749-762)Online publication date: Feb-2025
https://doi.org/10.1109/TMC.2024.3470339
Prakash RThomas T(2025)Towards Secure AI-driven Industrial Metaverse with NFT Digital Twins2025 17th International Conference on COMmunication Systems and NETworks (COMSNETS)10.1109/COMSNETS63942.2025.10885606(721-729)Online publication date: 6-Jan-2025
https://doi.org/10.1109/COMSNETS63942.2025.10885606
Li MBanerjee NBanerjee S(2025)Predicting 3D Motion from 2D Video for Behavior-Based VR Biometrics2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR63409.2025.00043(223-230)Online publication date: 27-Jan-2025
https://doi.org/10.1109/AIxVR63409.2025.00043
Show More Cited By

Index Terms

Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
2. Security and privacy
  1. Security services
    1. Authentication

Recommendations

VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

Haptic feedback in VR is important for realistic simulation in virtual reality. However, recreating the haptic experience for hand tools in VR traditionally requires hardware with precise actuators, adding complexity to the system. We propose Ungrounded ...
Virtually-Extended Proprioception: Providing Spatial Reference in VR through an Appended Virtual Limb
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Selecting targets directly in the virtual world is difficult due to the lack of haptic feedback and inaccurate estimation of egocentric distances. Proprioception, the sense of self-movement and body position, can be utilized to improve virtual target ...
Touch and Explore: A VR Game Exploration, Based on Haptic Driven Game-play
ISS Companion '21: Companion Proceedings of the 2021 Conference on Interactive Surfaces and Spaces

In digital games, visuals and audios are widely used to provide immersion; with an introduction to VR (Virtual Reality) in games, novel interaction opportunities are possible. Interactions in VR are similar to real-life. Hence haptics plays a crucial ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Deutsche Forschungsgemeinschaft

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

128
Total Citations
View Citations
2,960
Total Downloads

Downloads (Last 12 months)406
Downloads (Last 6 weeks)57

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu YZhang XWu TZhou BNguyen PLiu J(2025)3D Facial Tracking and User Authentication Through Lightweight Single-Ear BiosensorsIEEE Transactions on Mobile Computing10.1109/TMC.2024.347033924:2(749-762)Online publication date: Feb-2025
https://doi.org/10.1109/TMC.2024.3470339
Prakash RThomas T(2025)Towards Secure AI-driven Industrial Metaverse with NFT Digital Twins2025 17th International Conference on COMmunication Systems and NETworks (COMSNETS)10.1109/COMSNETS63942.2025.10885606(721-729)Online publication date: 6-Jan-2025
https://doi.org/10.1109/COMSNETS63942.2025.10885606
Li MBanerjee NBanerjee S(2025)Predicting 3D Motion from 2D Video for Behavior-Based VR Biometrics2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR63409.2025.00043(223-230)Online publication date: 27-Jan-2025
https://doi.org/10.1109/AIxVR63409.2025.00043
Jeon JNoh YKim JHong J(2025)Pre-AttentiveGaze: gaze-based authentication dataset with momentary visual interactionsScientific Data10.1038/s41597-025-04538-312:1Online publication date: 13-Feb-2025
https://doi.org/10.1038/s41597-025-04538-3
Renganathan BShanthi Priya RSenthil R(2024)Pre-Occupancy Evaluation of Affective Experiences in Building Information Modeling Through SLR and an e-Delphi SurveyTechnologies10.3390/technologies1301001413:1(14)Online publication date: 30-Dec-2024
https://doi.org/10.3390/technologies13010014
AlMutawa AIkuesan RSaid H(2024)Towards a Comprehensive Metaverse Forensic Framework Based on Technology Task Fit ModelFuture Internet10.3390/fi1612043716:12(437)Online publication date: 22-Nov-2024
https://doi.org/10.3390/fi16120437
Hallal LRhinelander JVenkat R(2024)Recent Trends of Authentication Methods in Extended Reality: A SurveyApplied System Innovation10.3390/asi70300457:3(45)Online publication date: 28-May-2024
https://doi.org/10.3390/asi7030045
Jiao ADuan DXu W(2024)Medusa3D: The Watchful Eye Freezing Illegitimate Users in Virtual Reality InteractionsProceedings of the ACM on Human-Computer Interaction10.1145/36765158:MHCI(1-21)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676515
Weiss YVilla SGrootjen JHoppe MKale YMüller F(2024)Exploring Redirection and Shifting Techniques to Mask Hand Movements from Shoulder-Surfing Attacks during PIN Authentication in Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36765028:MHCI(1-24)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676502
Pathmanathan NKurzhals K(2024)Investigating the Gap: Gaze and Movement Analysis in Immersive EnvironmentsProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653522(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653522
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten