Pivot: Panoramic-Image-Based VR User Authentication against Side-Channel Attacks
Authors: 
Gui Xiao, Zhen Ling, Qunqun Fan, Xiangyu Xu, Wenjia Wu, Ding Ding, Chen Chen, Xinwen FuAuthors Info & Claims
Gui Xiao, Zhen Ling, Qunqun Fan, Xiangyu Xu, Wenjia Wu, Ding Ding, Chen Chen, Xinwen FuAuthors Info & ClaimsArticle No.: 54, Pages 1 - 19
Abstract
With metaverse attracting increasing attention from both academic and industry, the application of virtual reality (VR) has extended beyond 3D immersive viewing/gaming to a broader range of areas, such as banking, shopping, tourism, education, and so on, which involves a growing amount of sensitive and private user data into VR systems. However, with current password-based user authentication schemes in mainstream VR devices, studies demonstrate that side-channel attacks can pose a severe threat to VR user privacy. To mitigate the threat, we propose a novel panoramic-image-based VR user authentication system, i.e., Pivot, to defend against such attacks, yet maintain high usability. Specifically, in Pivot, we design an image-random-pivoting-based user interaction mechanism to assist users in quickly and securely selecting memorable points of interest in a panoramic image. Then an image region segmentation algorithm is designed to automatically scatter the points to regions to form the customized graphic password for the user, which could ensure a sufficiently large password space and also reduce the near-region point misclicks. Afterward, the region indexes are used to generate the hashed password for authentication. Both theoretical security analysis and extensive user studies demonstrate that Pivot is secure and user-friendly in practice.
References
[1]
Abdullah Al Arafat, Zhishan Guo, and Amro Awad. 2021. VR-Spy: A side-channel attack on virtual key-logging in VR headsets. In Proceedings of the IEEE Virtual Reality and 3D User Interfaces (VR ’21). IEEE, Lisbon, Portugal, 564–572. DOI:
[2]
Merve Varol Ari̇soy and Ecir Uğur Küçüksille. 2021. Landmine Detection Training Simulation Using Virtual Reality Technology. Virtual Reality 25, 2 (2021), 461–490. DOI:
[3]
Bloomberg. 2021. Metaverse May Be $800 Billion Market, Next Tech Platform. Retrieved July 30, 2022 from https://www.bloomberg.com/professional/blog/metaverse-may-be-800-billion-market-next-tech-platform/
[4]
Sourour Chaabane, Anne-Marie Etienne, Michaël Schyns, and Aurélie Wagener. 2021. The Impact of Virtual Reality Exposure on Stress Level and Sense of Competence in Ambulance Workers. Journal of Traumatic Stress 35, 1 (2021), 120–127. DOI:
[5]
Yimin Chen, Tao Li, Rui Zhang, Yanchao Zhang, and Terri Hedgpeth. 2018. EyeTell: Video-assisted touchscreen keystroke inference from eye movements. In Proceedings of IEEE Symposium on Security and Privacy (S & P ’18). IEEE Computer Society, San Francisco, California, USA, 144–160. DOI:
[6]
Sonia Chiasson, Alain Forget, Robert Biddle, and Paul C. van Oorschot. 2008. Influencing Users Towards Better Passwords: Persuasive Cued Click-Points. People and Computers XXII Culture, Creativity, Interaction 1, 1 (2008), 121–130. DOI:
[7]
Sonia Chiasson, Paul C. van Oorschot, and Robert Biddle. 2007. Graphical password authentication using cued click points. In Proceedings of the 22nd European Symposium on Research in Computer Security (ESORICS ’07), Vol. 4734. Springer, Oslo, Norway, 359–374. DOI:
[8]
John P. Chin, Virginia A. Diehl, and Kent L. Norman. 1988. Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’88). ACM, Washington, D.C., USA, 213–218. DOI:
[9]
Wang Ding, Cheng Haibo, Wang Ping, Huang Xinyi, and Jian Gaopeng. 2017. Zipf’s Law in Passwords. IEEE Transactions on Information Forensics and Security 12, 11 (2017), 2776–2791. DOI:
[10]
Markus Funk, Karola Marky, Iori Mizutani, Mareike Kritzler, Simon Mayer, and Florian Michahelles. 2019. LookUnlock: Using spatial-targets for user-authentication on HMDs. In Proceedings of Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 1–6. DOI:
[11]
Daniel Genkin, Mihir Pattani, Roei Schuster, and Eran Tromer. 2019. Synesthesia: Detecting screen content via remote acoustic side channels. In Proceedings of IEEE Symposium on Security and Privacy (S & P ’19. IEEE, San Francisco, CA, USA, 853–869. DOI:
[12]
Ceenu George, Mohamed Khamis, Emanuel von Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017. Seamless and secure VR: Adapting and evaluating established authentication systems for virtual reality. In Proceedings of ISOC Network and Distributed System Security Symposium (NDSS ’17). ISOC, 1–15.
[13]
Ceenu George, Mohamed Khamis, Daniel Buschek, and Heinrich Hussmann. 2019. Investigating the third dimension for authentication in immersive virtual reality and in the real world. In Proceedings of the 27th IEEE Conference on Virtual Reality and 3d User Interfaces (VR ’19). IEEE, Osaka, Japan, 277–285. DOI:
[14]
Pascal Jansen and Fabian Fischbach. 2020. The social engineer: An immersive virtual reality educational game to raise social engineering awareness. In Proceedings of Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). ACM, Virtual Event, Canada, 59–63. DOI:
[15]
Huaizu Jiang, Jingdong Wang, Zejian Yuan, Yang Wu, Nanning Zheng, and Shipeng Li. 2013. Salient object detection: A discriminative regional feature integration approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVRP ’13). IEEE, Portland, OR, USA, 2083–2090. DOI:
[16]
William H. Kruskal and W. Allen Wallis. 1952. Use of Ranks in One-Criterion Variance Analysis. Journal of the American Statistical Association 47, 260 (1952), 583–621.
[17]
Sugang Li, Ashwin Ashok, Yanyong Zhang, Chenren Xu, Janne Lindqvist, and Macro Gruteser. 2016. Whose move is it anyway? Authenticating smart wearable devices using unique head movement patterns. In Proceedings of IEEE International Conference on Pervasive Computing and Communications (PerCom ’16). IEEE, Sydney, Australia, 1–9. DOI:
[18]
Zhen Ling, Zupei Li, Chen Chen, Junzhou Luo, Wei Yu, and Xinwen Fu. 2019. I know what you enter on gear VR. In Proceedings of IEEE 7th Conference on Communications and Network Security (CNS ’19). IEEE, Washington, DC, USA, 241–249. DOI:
[19]
Shiqing Luo, Xinyu Hu, and Zhisheng Yan. 2022. HoloLogger: Keystroke inference on mixed reality head mounted displays. In Proceedings of the 30th IEEE Conference on Virtual Reality and 3d User Interfaces (VR ’22). IEEE, Christchurch, New Zealand, 445–454. DOI:
[20]
Philipp Markert, Daniel V. Bailey, Maximilian Golla, Markus Dürmuth, and Adam J. Aviv. 2020. This pin can be easily guessed: Analyzing the security of smartphone unlock pins. In Proceedings of IEEE Symposium on Security and Privacy (S & P ’20). IEEE, San Francisco, CA, USA, 286–303. DOI:
[21]
Florian Mathis, Hassan Ismail Fawaz, and Mohamed Khamis. 2020. Knowledge-driven biometric authentication in virtual reality. In Proceedings of Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). ACM, Honolulu, HI, USA, 1–10. DOI:
[22]
Florian Mathis, John H. Williamson, Kami Vaniea, and Mohamed Khamis. 2021. Fast and Secure Authentication in Virtual Reality Using Coordinated 3d Manipulation and Pointing. ACM Transactions on Computer-Human Interaction 28, 1 (2021), 1–44. DOI:
[23]
Ülkü Meteriz-Yi̇ldi̇ran, Necip Fazi̇l Yi̇ldi̇ran, Amro Awad, and David Mohaisen. 2022. A keylogging inference attack on air-tapping keyboards in virtual environments. In Proceedings of the 30th IEEE Conference on Virtual Reality and 3d User Interfaces (VR ’22). IEEE, Christchurch, New Zealand, 765–774. DOI:
[24]
Robert Miller, Ashwin Ajit, Natasha Kholgade Banerjee, and Sean Banerjee. 2019. Realtime behavior-based continual authentication of users in virtual reality environments. In Proceedings of 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR ’19). IEEE, San Diego, CA, USA, 253–254. DOI:
[25]
Emiliano Miluzzo, Alexander Varshavsky, Suhrid Balakrishnan, and Romit Roy Choudhury. 2012. TapPrints: Your finger taps have fingerprints. In Proceedings of 10th International Conference on Mobile Systems, Applications, and Services (MobiSys ’12). ACM, Ambleside, United Kingdom, 323–336. DOI:
[26]
Ilesanmi Olade, Charles Fleming, and Hai-Ning Liang. 2020. BioMove: Biometric User Identification from Human Kinesiological Movements for Virtual Reality Systems. Sensors 20, 10 (2020), 29–44. DOI:
[27]
Vishnu Prajapati. 2018. Silhouette on People Standing on Mountain during Blue Hour. Retrieved from https://unsplash.com/photos/silhouette-on-people-standing-on-mountain-during-blue-hour-RaEFRWLy9ME
[28]
Samsung. 2019. The GearVR Framework (GearVRf). Retrieved July 30, 2022 from https://github.com/Samsung/GearVRf
[29]
Bob Shea. 2018. Seashore During Daytime. Retrieved from https://unsplash.com/photos/seashore-during-daytime-hIuCzCYx_pg
[30]
Yiran Shen, Hongkai Wen, Chengwen Luo, Weitao Xu, Tao Zhang, Wen Hu, and Daniela Rus. 2018. GaitLock: Protect Virtual and Augmented Reality Headsets Using Gait. IEEE Transactions on Dependable and Secure Computing 16, 3 (2018), 484–497. DOI:
[31]
Cong Shi, Xiangyu Xu, Tianfang Zhang, Payton Walker, Yi Wu, Jian Liu, Nitesh Saxena, Yingying Chen, and Jiadi Yu. 2021. Face-Mic: Inferring live speech and speaker identity via subtle facial dynamics captured by AR/VR motion sensors. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (MobiCom ’21). ACM, New Orleans, Louisiana, USA, 478–490. DOI:
[32]
Shailesh Kumar Shivakumar and Sourabhh Sethii. 2019. Transforming Legacy Banking Applications to Banking Experience Platforms. Springer, 261–295.
[33]
Jingchao Sun, Xiaocong Jin, Yimin Chen, Jinxue Zhang, Rui Zhang, and Yanchao Zhang. 2016. Visible: Video-assisted keystroke inference from tablet backside motion. In Proceedings of ISOC Network and Distributed System Security Symposium (NDSS ’16). ISOC, San Diego, California, USA, 1–15. DOI:
[34]
David Vives. 2020. Aerial View of City Buildings near Sea during Daytime. Retrieved from https://unsplash.com/photos/aerial-view-of-city-buildings-near-sea-during-daytime-VP-Xs6MF0Fk
[35]
Ding Wang, Qianchen Gu, Xinyi Huang, and Ping Wang. 2017. Understanding human-chosen PINs: Characteristics, distribution and security. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security (CCS ’17). ACM, New York, NY, 372–385. DOI:
[36]
Susan Wiedenbeck, Jim Waters, Jean-Camille Birget, Alex Brodskiy, and Nasir Memon. 2005. PassPoints: Design and Longitudinal Evaluation of a Graphical Password System. International Journal of Human-computer Studies 63, 1–2 (2005), 102–127. DOI:
[37]
Wikipedia. 2022. Equirectangular Projection. Retrieved July 30, 2022 from https://en.wikipedia.org/wiki/Equirectangular_projection
[38]
Yi Xu, Jared Heinly, Andrew M. White, Fabian Monrose, and Jan-Michael Frahm. 2013. Seeing double: Reconstructing obscured typed input from repeated compromising reflections. In Proceedings of 2013 ACM SIGSAC Conference on Computer and Communications Security (CCS ’13). ACM, Berlin, Germany, 1063–1074. DOI:
[39]
Guixin Ye, Zhanyong Tang, Dingyi Fang, Xiaojiang Chen, Kwang In Kim, Ben Taylor, and Zheng Wang. 2017. Cracking android pattern lock in five attempts. In Proceedings of ISOC Network and Distributed System Security Symposium (NDSS ’17). ISOC, San Diego, California, USA, 1–15. DOI:
[40]
Yevheniia. 2021. A View of a City and a Body of Water. Retrieved from https://unsplash.com/photos/a-view-of-a-city-and-a-body-of-water-ZMuldmPnOOI
[41]
Zhen Yu, Hai-Ning Liang, Charles Fleming, and Ka Lok Man. 2016. An exploration of usable authentication mechanisms for virtual reality systems. In Proceedings of 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS ’16). IEEE, Jeju, South Korea, 458–460. DOI:
[42]
Qinggang Yue, Zhen Ling, Xinwen Fu, Benyuan Liu, Kui Ren, and Wei Zhao. 2014. Blind recognition of touched keys on mobile devices. In Proceedings of 2014 ACM SIGSAC Conference on Computer and Communications Security (CCS ’14). ACM, Scottsdale, AZ, USA, 1403–1414. DOI:
[43]
Man Zhou, Qian Wang, Jingxiao Yang, Qi Li, Feng Xiao, Zhibo Wang, and Xiaofeng Chen. 2018. PatternListener: Cracking android pattern lock using acoustic signals. In Proceedings of the 14th ACM Conference on Computer and Communications Security (CCS ’18). ACM, Toronto, ON, Canada, 1775–1787. DOI:
[44]
Huadi Zhu, Mingyan Xiao, Demoria Sherman, and Ming Li. 2023. SoundLock: A novel user authentication scheme for VR devices using auditory-pupillary response. In Proceedings of ISOC Network and Distributed System Security Symposium (NDSS ’23). ISOC, 1–15. DOI:
[45]
Mark Zuckerberg and Gayle King. 2021. Facebook Launches “Horizon Workrooms.” Here’s How It Works. Retrieved February 15, 2022 from https://www.cbsnews.com/video/facebook-launches-horizon-workrooms-heres-how-it-works/
Index Terms
- Pivot: Panoramic-Image-Based VR User Authentication against Side-Channel Attacks
Recommendations
Prefetch Side-Channel Attacks: Bypassing SMAP and Kernel ASLR
CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications SecurityModern operating systems use hardware support to protect against control-flow hijacking attacks such as code-injection attacks. Typically, write access to executable pages is prevented and kernel mode execution is restricted to kernel code pages only. ...
Securing Memory Encryption and Authentication Against Side-Channel Attacks Using Unprotected Primitives
ASIA CCS '17: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications SecurityMemory encryption is used in many devices to protect memory content from attackers with physical access to a device. However, many current memory encryption schemes can be broken using Differential Power Analysis (DPA). In this work, we present MEAS---...
Security beyond cybersecurity: side-channel attacks against non-cyber systems and their countermeasures
AbstractSide-channels are unintended pathways within target systems that leak internal information, exploitable via side-channel attack techniques that extract the target information, compromising the system’s security and privacy. Side-channel attacks ...
Comments
Information & Contributors
Information
Published In
February 2025
651 pages
EISSN:1551-6865
DOI:10.1145/3703007
- Editor:
- Abuabdulmotaleb El Saddik,
- Guest Editorss:
- Yushu Zhang,
- William Puech,
- Anderson Rocha,
- Rongxing Lu,
- Stefano Cresci,
- Roberto Di Pietro
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 January 2025
Online AM: 09 September 2024
Accepted: 20 August 2024
Revised: 19 July 2024
Received: 30 December 2023
Published in TOMM Volume 21, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- US National Science Foundation (NSF)
- Jiangsu Provincial Natural Science Foundation of China
- Jiangsu Provincial Key R&D Programs
- ”Zhishan” Young Scholar Program of Southeast University
- Jiangsu Provincial Key Laboratory of Network and Information Security
- Key Laboratory of Computer Network and Information Integration of Ministry of Education of China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 117Total Downloads
- Downloads (Last 12 months)117
- Downloads (Last 6 weeks)28
Reflects downloads up to 15 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in