Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3607199.3607209acmotherconferencesArticle/Chapter ViewAbstractPublication PagesraidConference Proceedingsconference-collections
research-article

SigA: rPPG-based Authentication for Virtual Reality Head-mounted Display

Published: 16 October 2023 Publication History

Abstract

Consumer-grade virtual reality head-mounted displays (VR-HMD) are becoming increasingly popular. Despite VR’s convenience and booming applications, VR-based authentication schemes are underdeveloped. The recently proposed authentication methods (Electrooculogram based, Electrical Muscle Stimulation-based, and alike) require active user involvement, disturbing many scenarios like drone flight and telemedicine. This paper proposes an effective and efficient user authentication method in VR environments resilient to impersonation attacks using physiological signals — Photoplethysmogram (PPG), namely SigA. SigA exploits the advantage that PPG is a physiological signal invisible to the naked eye. Using VR-HMDs to cover the eye area completely, SigA reduces the risk of signal leakage during PPG acquisition. We conducted a comprehensive analysis of SigA’s feasibility on five publicly available datasets, nine different pre-trained models, three facial regions, various lengths of the video clips required for training, four different signal time intervals, and continuous authentication with different sliding window sizes. The results demonstrate that SigA achieves more than 95% of the average F1-score in a one-second signal to accommodate a complete cardiac cycle for most adults, implying its applicability in real-world scenarios. Furthermore, experiments have shown that SigA is resistant to zero-effort attacks, statistical attacks, impersonation attacks (with a detection accuracy of over 95%) and session hijacking attacks.

References

[1]
Yasmeen Abdrabou, Sheikh Radiah Rivu, Tarek Ammar, Jonathan Liebers, Alia Saad, Carina Liebers, Uwe Gruenefeld, Pascal Knierim, Mohamed Khamis, Ville Makela, 2022. Understanding shoulder surfer behavior and attack patterns using virtual reality. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces. 1–9.
[2]
Andy Adler, Richard Youmaran, and Sergey Loyka. 2009. Towards a measure of biometric feature information. Pattern Analysis and Applications 12, 3 (2009), 261–270.
[3]
Abhishek Aich, Meng Zheng, Srikrishna Karanam, Terrence Chen, Amit K Roy-Chowdhury, and Ziyan Wu. 2021. Spatio-temporal representation factorization for video-based person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 152–162.
[4]
Serge Bobbia, Richard Macwan, Yannick Benezeth, Alamin Mansouri, and Julien Dubois. 2019. Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognition Letters 124 (2019), 82–90.
[5]
Giuseppe Boccignone, Donatello Conte, Vittorio Cuculo, Alessandro D’Amelio, Giuliano Grossi, Raffaella Lanzarotti, and Edoardo Mortara. 2022. pyVHR: A Python framework for remote photoplethysmography. PeerJ Computer Science 8 (2022), e929.
[6]
Tananant Boonya-Ananta, Andres J Rodriguez, Ajmal Ajmal, Vinh Nguyen Du Le, Anders K Hansen, Joshua D Hutcheson, and Jessica C Ramella-Roman. 2021. Synthetic photoplethysmography (PPG) of the radial artery through parallelized Monte Carlo and its correlation to body mass index (BMI). Scientific reports 11, 1 (2021), 1–11.
[7]
Deivid Botina-Monsalve, Yannick Benezeth, and Johel Miteran. 2022. RTrPPG: An ultra light 3DCNN for real-time remote photoplethysmography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2146–2154.
[8]
Diane Bouchacourt, Mark Ibrahim, and Ari Morcos. 2021. Grounding inductive biases in natural images: Invariance stems from variations in data. Advances in Neural Information Processing Systems 34 (2021), 19566–19579.
[9]
Ayan Chakraborty, Dharitri Goswami, Jayanta Mukhopadhyay, and Saswat Chakrabarti. 2020. Measurement of arterial blood pressure through single-site acquisition of photoplethysmograph signal. IEEE Transactions on Instrumentation and Measurement 70 (2020), 1–10.
[10]
Weixuan Chen and Daniel McDuff. 2018. Deepphys: Video-based physiological measurement using convolutional attention networks. In Proceedings of the European Conference on Computer Vision (ECCV). 349–365.
[11]
Yuxin Chen, Zhuolin Yang, Ruben Abbou, Pedro Lopes, Ben Y Zhao, and Haitao Zheng. 2021. User authentication via electrical muscle stimulation. In Proceedings of the Conference on Human Factors in Computing Systems. 1–15.
[12]
Thomas M Cover, Joy A Thomas, 1991. Entropy, relative entropy and mutual information. Elements of Information Theory 2, 1 (1991), 12–13.
[13]
Jamie Ian Cross, Christine Boag-Hodgson, Tim Ryley, Timothy Mavin, and Leigh Ellen Potter. 2022. Using extended reality in flight simulators: A literature review. IEEE Transactions on Visualization and Computer Graphics (2022).
[14]
Gerard De Haan and Vincent Jeanne. 2013. Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering 60, 10 (2013), 2878–2886.
[15]
Ruggero Donida Labati, Vincenzo Piuri, Francesco Rundo, Fabio Scotti, and Concetto Spampinato. 2021. Biometric recognition of PPG cardiac signals using transformed spectrogram images. In Proceedings of the ICPR Workshop on Mobile and Wearable Biometrics (WMWB), Vol. 12668. Springer, 244–257.
[16]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the 2021 International Conference on Learning Representations (ICLR).
[17]
Timothy Dozat. 2016. Incorporating nesterov momentum into adam. In Proceedings of the International Conference on Learning Representations Workshop. 1–4.
[18]
Reyhan Düzgün, Naheem Noah, Peter Mayer, Sanchari Das, and Melanie Volkamer. 2022. SoK: A systematic literature review of knowledge-based authentication on augmented reality head-mounted displays. In Proceedings of the 17th International Conference on Availability, Reliability and Security. 1–12.
[19]
Simon Eberz, Nicola Paoletti, Marc Roeschlin, Marta Kwiatkowska, I Martinovic, and A Patané. 2017. Broken hearted: How to attack ECG biometrics. In Proceedings of the Network and Distributed System Security Symposium (NDSS ’17).
[20]
Mohamed Elgendi. 2012. On the analysis of fingertip photoplethysmogram signals. Current cardiology reviews 8, 1 (2012), 14–25.
[21]
Markus Funk, Karola Marky, Iori Mizutani, Mareike Kritzler, Simon Mayer, and Florian Michahelles. 2019. Lookunlock: Using spatial-targets for user-authentication on hmds. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.
[22]
Ceenu George, Mohamed Khamis, Emanuel von Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017. Seamless and secure VR: Adapting and evaluating established authentication systems for virtual reality. In Proceedings of the Network and Distributed System Security Symposium (NDSS).
[23]
Martin Georgiev, Simon Eberz, Henry Turner, Giulio Lovisotto, and Ivan Martinovic. 2022. Common evaluation pitfalls in touch-based authentication systems. In Proceedings of the ACM on Asia Conference on Computer and Communications Security. 1049–1063.
[24]
Grand View Research. 2022. Virtual Reality Market Size, Share & Trends Analysis Report by Technology (Semi & Fully Immersive, Non-immersive), By Device (HMD, GTD), By Component (Hardware, Software), By Application, And Segment Forecasts, 2022–2030. Technical Report. GVR-1-68038-831-2. Retrieved Jan 29, 2023 from https://www.grandviewresearch.com/industry-analysis/virtual-reality-vr-market
[25]
Y.Y. Gu, Y. Zhang, and Y.T. Zhang. 2003. A novel biometric approach in human verification by photoplethysmographic signals. In Proceedings of the 4th International IEEE EMBS Special Topic Conference on Information Technology Applications in Biomedicine. 13–14.
[26]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[27]
Guillaume Heusch, André Anjos, and Sébastien Marcel. 2017. A reproducible study on remote heart rate measurement. arXiv preprint arXiv:1709.00962 (2017).
[28]
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4700–4708.
[29]
Dae Yon Hwang, Bilal Taha, and Dimitrios Hatzinakos. 2021. PBGAN: Learning PPG Representations from GAN for Time-Stable and Unique Verification System. IEEE Transactions on Information Forensics and Security 16 (2021), 5124–5137.
[30]
Seol-Jung Kang, Gi-Chul Ha, and Kwang-Jun Ko. 2017. Association between resting heart rate, metabolic syndrome and cardiorespiratory fitness in Korean male adults. Journal of Exercise Science & Fitness 15, 1 (2017), 27–31.
[31]
Yury Kartynnik, Artsiom Ablavatski, Ivan Grishchenko, and Matthias Grundmann. 2019. Real-time facial surface geometry from monocular video on mobile GPUs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
[32]
A. Reşit Kavsaoğlu, Kemal Polat, and M. Recep Bozkurt. 2014. A novel feature ranking algorithm for biometric recognition with PPG signals. Computers in Biology and Medicine 49 (2014), 1–14.
[33]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012).
[34]
Lin Li, Chao Chen, Lei Pan, Jun Zhang, and Yang Xiang. 2022. Video is all you need: Attacking PPG-based biometric authentication. In Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security. 57–66.
[35]
Yongbo Liang, Zhencheng Chen, Guiyong Liu, and Mohamed Elgendi. 2018. A new, short-recorded photoplethysmogram dataset for blood pressure monitoring in China. Scientific data 5, 1 (2018), 1–7.
[36]
Feng Lin, Kun Woo Cho, Chen Song, Wenyao Xu, and Zhanpeng Jin. 2018. Brain password: A secure and truly cancelable brain biometrics for smart headwear. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ’18). 296–309.
[37]
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the 2017 IEEE International Conference on Computer Vision. 2980–2988.
[38]
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022).
[39]
Shiqing Luo, Anh Nguyen, Chen Song, Feng Lin, Wenyao Xu, and Zhisheng Yan. 2020. OcuLock: Exploring human visual system for authentication in virtual reality head-mounted display. In Proceedings of the Network and Distributed System Security Symposium (NDSS).
[40]
Jesus Martínez-Navarro, Enrique Bigné, Jaime Guixeres, Mariano Alcañiz, and Carmen Torrecilla. 2019. The influence of virtual reality in e-commerce. Journal of Business Research 100 (2019), 475–482.
[41]
Florian Mathis, Hassan Ismail Fawaz, and Mohamed Khamis. 2020. Knowledge-driven biometric authentication in virtual reality. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–10.
[42]
Florian Mathis, John Williamson, Kami Vaniea, and Mohamed Khamis. 2020. Rubikauth: Fast and secure authentication in virtual reality. In Extended Abstracts of the Conference on Human Factors in Computing Systems. 1–9.
[43]
Massimiliano Mazza, Kornelius Kammler-Sücker, Tagrid Leménager, Falk Kiefer, and Bernd Lenz. 2021. Virtual reality: A powerful technology to provide novel insight into treatment mechanisms of addiction. Translational Psychiatry 11, 1 (2021), 1–11.
[44]
Daniel McDuff. 2023. Camera measurement of physiological vital signs. Comput. Surveys 55, 9 (2023), 1–40.
[45]
Rita Meziatisabour, Yannick Benezeth, Pierre De Oliveira, Julien Chappe, and Fan Yang. 2021. UBFC-Phys: A multimodal database for psychophysiological studies of social stress. IEEE Transactions on Affective Computing (2021).
[46]
Genki Nagamatsu, Ewa Magdalena Nowara, Amruta Pai, Ashok Veeraraghavan, and Hiroshi Kawasaki. 2020. PPG3D: Does 3D head tracking improve camera-based PPG estimation?. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 1194–1197.
[47]
Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. 2020. What is being transferred in transfer learning?Advances in Neural Information Processing Systems 33 (2020), 512–523.
[48]
Xuesong Niu, Shiguang Shan, Hu Han, and Xilin Chen. 2019. Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation. IEEE Transactions on Image Processing 29 (2019), 2409–2423.
[49]
Soyeon Park, Bo-Kyeong Kim, and Suh-Yeon Dong. 2022. Self-supervised RGB-NIR fusion video vision transformer framework for rPPG estimation. IEEE Transactions on Instrumentation and Measurement 71 (2022), 1–10.
[50]
Christian S. Pilz, Sebastian Zaunseder, Jarek Krajewski, and Vladimir Blazek. 2018. Local group invariance for heart rate estimation from face videos in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1254–1262.
[51]
Napa Sae-Bae and Nasir Memon. 2022. Distinguishability of keystroke dynamic template. Plos One 17, 1 (2022), e0261291.
[52]
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4510–4520.
[53]
Stefan Schneegass, Youssef Oualil, and Andreas Bulling. 2016. SkullConduct: Biometric user identification on eyewear computers using bone conduction through the skull. In Proceedings of the Conference on Human Factors in Computing Systems. 1379–1384.
[54]
Abdul Serwadda and Vir V Phoha. 2013. Examining a large keystroke biometrics dataset for statistical-attack openings. ACM Transactions on Information and System Security (TISSEC) 16, 2 (2013), 1–30.
[55]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[56]
Rencheng Song, Senle Zhang, Juan Cheng, Chang Li, and Xun Chen. 2020. New insights on super-high resolution for video-based heart rate estimation with a semi-blind source separation method. Computers in biology and medicine 116 (2020), 103535.
[57]
Sophie Stephenson, Bijeeta Pal, Stephen Fan, Earlence Fernandes, Yuhang Zhao, and Rahul Chatterjee. 2022. SoK: Authentication in augmented and virtual reality. In Proceedings of the IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 1552–1552.
[58]
Nicolette Stogios, Alexander Gdanski, Philip Gerretsen, Araba F Chintoh, Ariel Graff-Guerrero, Tarek K Rajji, Gary Remington, Margaret K Hahn, and Sri Mahavir Agarwal. 2021. Autonomic nervous system dysfunction in schizophrenia: Impact on cognitive and metabolic health. NPJ Schizophrenia 7, 1 (2021), 1–10.
[59]
Ronny Stricker, Steffen Müller, and Horst-Michael Gross. 2014. Non-contact video-based pulse rate measurement on a mobile service robot. In Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 1056–1062.
[60]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–9.
[61]
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2818–2826.
[62]
Kenta Takahashi and Takao Murakami. 2014. A measure of information gained through biometric systems. Image and Vision Computing 32, 12 (2014), 1194–1203.
[63]
Wim Verkruysse, Lars O Svaasand, and J Stuart Nelson. 2008. Remote plethysmographic imaging using ambient light. Optics Express 16, 26 (2008), 21434–21445.
[64]
Ding Wang, Haibo Cheng, Ping Wang, Xinyi Huang, and Gaopeng Jian. 2017. Zipf’s law in passwords. IEEE Transactions on Information Forensics and Security 12, 11 (2017), 2776–2791.
[65]
Ding Wang, Qianchen Gu, Xinyi Huang, and Ping Wang. 2017. Understanding human-chosen pins: Characteristics, distribution and security. In Proceedings of the ACM on Asia Conference on Computer and Communications Security. 372–385.
[66]
Haohuang Wen, Phillip Porras, Vinod Yegneswaran, and Zhiqiang Lin. 2023. Thwarting smartphone SMS attacks at the radio interface layer. In Proceedings of the Network and Distributed System Security Symposium (NDSS ’23).
[67]
Hui Yen Yap, Yun-Huoy Choo, Zeratul Izzah Mohd Yusoh, and Wee How Khoh. 2021. Person authentication based on eye-closed and visual stimulation using EEG signals. Brain Informatics 8, 1 (2021), 1–13.
[68]
Richard Youmaran and Andy Adler. 2012. Measuring biometric sample quality in terms of biometric feature information in iris images. Journal of Electrical and Computer Engineering 2012 (2012), 22–22.
[69]
Zhen Yu, Hai-Ning Liang, Charles Fleming, and Ka Lok Man. 2016. An exploration of usable authentication mechanisms for virtual reality systems. In Proceedings of the IEEE Asia Pacific Conference on Circuits and Systems (APCCAS). IEEE, 458–460.
[70]
Tianming Zhao, Yan Wang, Jian Liu, Yingying Chen, Jerry Cheng, and Jiadi Yu. 2020. Trueheart: Continuous authentication on wrist-worn wearables using PPG-based biometrics. In Proceedings of the Annual IEEE International Conference on Computer Communications (INFOCOM). 30–39.
[71]
Huadi Zhu, Mingyan Xiao, Demoria Sherman, and Ming Li. 2023. SoundLock: ANovel user authentication scheme for VR devices using auditory-pupillary response. In Proceedings of the Network and Distributed System Security Symposium (NDSS).
[72]
Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Advances in neural information processing systems 32 (2019).
[73]
Tiantian Zhu, Lei Fu, Qiang Liu, Zi Lin, Yan Chen, and Tieming Chen. 2020. One cycle attack: Fool sensor-based personal gait authentication with clustering. IEEE Transactions on Information Forensics and Security 16 (2020), 553–568.
[74]
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. 2020. A comprehensive survey on transfer learning. Proc. IEEE 109, 1 (2020), 43–76.

Cited By

View all
  • (2024)Non-Contact Vision-Based Techniques of Vital Sign Monitoring: Systematic ReviewSensors10.3390/s2412396324:12(3963)Online publication date: 19-Jun-2024
  • (2024)Camera-based physiological measurementNeurocomputing10.1016/j.neucom.2024.127282575:COnline publication date: 16-May-2024

Index Terms

  1. SigA: rPPG-based Authentication for Virtual Reality Head-mounted Display

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      RAID '23: Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses
      October 2023
      769 pages
      ISBN:9798400707650
      DOI:10.1145/3607199
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 16 October 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Biometric Authentication
      2. Head-mounted Display
      3. Photoplethysmography
      4. Physiological Signal
      5. Virtual Reality

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      RAID 2023

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)176
      • Downloads (Last 6 weeks)11
      Reflects downloads up to 12 Sep 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Non-Contact Vision-Based Techniques of Vital Sign Monitoring: Systematic ReviewSensors10.3390/s2412396324:12(3963)Online publication date: 19-Jun-2024
      • (2024)Camera-based physiological measurementNeurocomputing10.1016/j.neucom.2024.127282575:COnline publication date: 16-May-2024

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media