Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3411764.3445478acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

RepliCueAuth: Validating the Use of a Lab-Based Virtual Reality Setup for Evaluating Authentication Systems

Published: 07 May 2021 Publication History

Abstract

Evaluating novel authentication systems is often costly and time-consuming. In this work, we assess the suitability of using Virtual Reality (VR) to evaluate the usability and security of real-world authentication systems. To this end, we conducted a replication study and built a virtual replica of CueAuth [52], a recently introduced authentication scheme, and report on results from: (1) a lab-based in-VR usability study (N=20) evaluating user performance; (2) an online security study (N=22) evaluating system’s observation resistance through virtual avatars; and (3) a comparison between our results and those previously reported in the real-world evaluation. Our analysis indicates that VR can serve as a suitable test-bed for human-centred evaluations of real-world authentication schemes, but the used VR technology can have an impact on the evaluation. Our work is a first step towards augmenting the design and evaluation spectrum of authentication systems and offers ground work for more research to follow.

Supplementary Material

Supplementary Materials (3411764.3445478_supplementalmaterials.zip)
MP4 File (3411764.3445478_videofigure.mp4)
Supplemental video
MP4 File (3411764.3445478_videopreview.mp4)
Preview video

References

[1]
Yasmeen Abdrabou, Mohamed Khamis, Rana Mohamed Eisa, Sherif Ismail, and Amrl Elmougy. 2019. Just Gaze and Wave: Exploring the Use of Gaze and Gestures for Shoulder-Surfing Resilient Authentication. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications (Denver, Colorado) (ETRA ’19). Association for Computing Machinery, New York, NY, USA, Article 29, 10 pages. https://doi.org/10.1145/3314111.3319837
[2]
John G Adair. 1984. The Hawthorne effect: a reconsideration of the methodological artifact.Journal of applied psychology 69, 2 (1984), 334. https://doi.org/10.1037/0021-9010.69.2.334
[3]
Sara Albakry, Kami Vaniea, and Maria K. Wolters. 2020. What is This URL’s Destination? Empirical Evaluation of Users’ URL Reading. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376168
[4]
Dmitry Alexandrovsky, Susanne Putze, Michael Bonfert, Sebastian Höffner, Pitt Michelmann, Dirk Wenig, Rainer Malaka, and Jan David Smeddinck. 2020. Examining Design Choices of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–21. https://doi.org/10.1145/3313831.3376260
[5]
K. Althobaiti, G. Rummani, and K. Vaniea. 2019. A Review of Human- and Computer-Facing URL Phishing Features. In 2019 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). EuroS&PW, Stockholm, Sweden, 182–191. https://doi.org/10.1109/EuroSPW.2019.00027
[6]
Ilhan Aslan, Andreas Uhl, Alexander Meschtscherjakov, and Manfred Tscheligi. 2014. Mid-Air Authentication Gestures: An Exploration of Authentication Based on Palm and Finger Motions. In Proceedings of the 16th International Conference on Multimodal Interaction (Istanbul, Turkey) (ICMI ’14). Association for Computing Machinery, New York, NY, USA, 311–318. https://doi.org/10.1145/2663204.2663246
[7]
Adam J. Aviv, John T. Davin, Flynn Wolf, and Ravi Kuber. 2017. Towards Baselines for Shoulder Surfing on Mobile Authentication. In Proceedings of the 33rd Annual Computer Security Applications Conference (Orlando, FL, USA) (ACSAC 2017). Association for Computing Machinery, New York, NY, USA, 486–498. https://doi.org/10.1145/3134600.3134609
[8]
Adam J. Aviv, Flynn Wolf, and Ravi Kuber. 2018. Comparing Video Based Shoulder Surfing with Live Simulation. In Proceedings of the 34th Annual Computer Security Applications Conference (San Juan, PR, USA) (ACSAC ’18). Association for Computing Machinery, New York, NY, USA, 453–466. https://doi.org/10.1145/3274694.3274702
[9]
Louise Barkhuus and Jennifer A. Rode. 2007. From Mice to Men - 24 Years of Evaluation in CHI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 1. https://doi.org/10.1145/1240624.2180963
[10]
Philip AE Brey. 2011. Anticipatory technology ethics for emerging IT. CEPE 2011: Crossing Boundaries 6 (2011), 13.
[11]
Kelly Caine. 2016. Local Standards for Sample Size at CHI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 981–992. https://doi.org/10.1145/2858036.2858498
[12]
Victoria Chang, Pramod Chundury, and Marshini Chetty. 2017. Spiders in the Sky: User Perceptions of Drones, Privacy, and Security. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 6765–6776. https://doi.org/10.1145/3025453.3025632
[13]
Scott Clifford and Jennifer Jerit. 2014. Is There a Cost to Convenience? An Experimental Comparison of Data Quality in Laboratory and Online Studies. Journal of Experimental Political Science 1, 2 (2014), 120–131. https://doi.org/10.1017/xps.2014.5
[14]
Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science 349, 6251 (2015), 13. https://doi.org/10.1126/science.aac4716
[15]
Jessica Colnago, Summer Devlin, Maggie Oates, Chelse Swoopes, Lujo Bauer, Lorrie Cranor, and Nicolas Christin. 2018. “It’s Not Actually That Horrible”: Exploring Adoption of Two-Factor Authentication at a University. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3173574.3174030
[16]
Frédéric Dandurand, Thomas R Shultz, and Kristine H Onishi. 2008. Comparing online and lab methods in a problem-solving experiment. Behavior research methods 40, 2 (2008), 428–434. https://doi.org/10.3758/BRM.40.2.428
[17]
Beymer David and Flickner Myron. 2003. Eye gaze tracking using an active stereo head. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., Vol. 2. IEEE Computer Society, USA, II–451. https://doi.org/10.1109/CVPR.2003.1211502
[18]
Alexander De Luca, Marian Harbach, Emanuel von Zezschwitz, Max-Emanuel Maurer, Bernhard Ewald Slawik, Heinrich Hussmann, and Matthew Smith. 2014. Now You See Me, Now You Don’t: Protecting Smartphone Authentication from Shoulder Surfers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 2937–2946. https://doi.org/10.1145/2556288.2557097
[19]
Alexander De Luca, Emanuel von Zezschwitz, Ngo Dieu Huong Nguyen, Max-Emanuel Maurer, Elisa Rubegni, Marcello Paolo Scipioni, and Marc Langheinrich. 2013. Back-of-Device Authentication on Smartphones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 2389–2398. https://doi.org/10.1145/2470654.2481330
[20]
Alexander De Luca, Roman Weiss, and Heiko Drewes. 2007. Evaluation of Eye-Gaze Interaction Methods for Security Enhanced PIN-Entry. In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces (Adelaide, Australia) (OZCHI ’07). Association for Computing Machinery, New York, NY, USA, 199–202. https://doi.org/10.1145/1324892.1324932
[21]
Nicola Dell, Vidya Vaidyanathan, Indrani Medhi, Edward Cutrell, and William Thies. 2012. ”Yours is Better!”: Participant Response Bias in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1321–1330. https://doi.org/10.1145/2207676.2208589
[22]
Statista Research Department. 2019. Share of respondents who own a virtual reality headset computing device in the United Kingdom in 2019, by generation. https://www.statista.com/statistics/1044070/uk-virtual-reality-headset-ownership/ accessed 15 September 2020.
[23]
Alan Dix, Janet Finlay, Gregory D Abowd, and Russell Beale. 2003. Human-computer interaction. Pearson Education, USA.
[24]
Sascha Fahl, Marian Harbach, Yasemin Acar, and Matthew Smith. 2013. On the Ecological Validity of a Password Study. In Proceedings of the Ninth Symposium on Usable Privacy and Security (Newcastle, United Kingdom) (SOUPS ’13). Association for Computing Machinery, New York, NY, USA, Article 13, 13 pages. https://doi.org/10.1145/2501604.2501617
[25]
Timothy J. Forman, Daniel S. Roche, and Adam J. Aviv. 2020. Twice as Nice? A Preliminary Evaluation of Double Android Unlock Patterns. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3334480.3382922
[26]
Alexander Gamero-Garrido, Stefan Savage, Kirill Levchenko, and Alex C. Snoeren. 2017. Quantifying the Pressure of Legal Risks on Third-Party Vulnerability Research. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security(Dallas, Texas, USA) (CCS ’17). Association for Computing Machinery, New York, NY, USA, 1501–1513. https://doi.org/10.1145/3133956.3134047
[27]
Simson Garfinkel and Heather Richter Lipford. 2014. Usable security: History, themes, and challenges. Synthesis Lectures on Information Security, Privacy, and Trust 5, 2(2014), 1–124.
[28]
Ceenu George, Daniel Buschek, Andrea Ngao, and Mohamed Khamis. 2020. GazeRoomLock: Using Gaze and Head-pose to Improve the Usability and ObservationResistance of 3D Passwords in Virtual Reality. In Augmented Reality, Virtual Reality, and Computer Graphics. Springer International Publishing. https://doi.org/10.1007/978-3-030-58465-8_5
[29]
Ceenu George, Mohamed Khamis, Emanuel von Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017. Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality. In Network and Distributed System Security Symposium (NDSS 2017)(USEC ’17). NDSS, 12. https://doi.org/10.14722/usec.2017.23028
[30]
Yiannis Georgiou and Eleni A. Kyza. 2017. The development and validation of the ARI questionnaire: An instrument for measuring immersion in location-based augmented reality settings. International Journal of Human-Computer Studies 98 (2017), 24 – 37. https://doi.org/10.1016/j.ijhcs.2016.09.014
[31]
Lisa M Given. 2008. The Sage encyclopedia of qualitative research methods. Sage publications. https://doi.org/10.4135/9781412963909.n381
[32]
Gene V Glass, Percy D. Peckham, and James R. Sanders. 1972. Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance. Review of Educational Research 42, 3 (1972), 237–288. https://doi.org/10.3102/00346543042003237
[33]
Dominic Gorecky, Mohamed Khamis, and Katharina Mura. 2017. Introduction and establishment of virtual training in the factory of the future. International Journal of Computer Integrated Manufacturing 30, 1(2017), 182–190. https://doi.org/10.1080/0951192X.2015.1067918
[34]
Saul Greenberg and Bill Buxton. 2008. Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 111–120. https://doi.org/10.1145/1357054.1357074
[35]
Stephan Guttinger. 2020. The limits of replicability. European Journal for Philosophy of Science 10, 2 (2020), 10. https://doi.org/10.1007/s13194-019-0269-1
[36]
Christopher Hadnagy. 2010. Social engineering: The art of human hacking. John Wiley & Sons.
[37]
Marian Harbach, Emanuel Von Zezschwitz, Andreas Fichtner, Alexander De Luca, and Matthew Smith. 2014. It’s a Hard Lock Life: A Field Study of Smartphone (Un)Locking Behavior and Risk Perception. In Proceedings of the Tenth USENIX Conference on Usable Privacy and Security (Menlo Park, CA) (SOUPS ’14). USENIX Association, USA, 213–230.
[38]
Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage publications Sage CA: Los Angeles, CA, 904–908. https://doi.org/10.1177/154193120605000909
[39]
Marc Hassenzahl, Michael Burmester, and Franz Koller. 2003. AttrakDiff: A questionnaire to measure perceived hedonic and pragmatic quality. In Mensch & Computer, Vol. 57. 187–196.
[40]
Eiji Hayashi, Oriana Riva, Karin Strauss, A. J. Bernheim Brush, and Stuart Schechter. 2012. Goldilocks and the Two Mobile Devices: Going beyond All-or-Nothing Access to a Device’s Applications. In Proceedings of the Eighth Symposium on Usable Privacy and Security (Washington, D.C.) (SOUPS ’12). Association for Computing Machinery, New York, NY, USA, Article 2, 11 pages. https://doi.org/10.1145/2335356.2335359
[41]
Jerry L. Hintze and Ray D. Nelson. 1998. Violin Plots: A Box Plot-Density Trace Synergism. The American Statistician 52, 2 (1998), 181–184. https://doi.org/10.1080/00031305.1998.10480559
[42]
HTC. 2016. HTC VIVE Pro Eye. https://www.vive.com/uk/product/vive-pro-eye/ accessed 17 August 2020.
[43]
HaptX Inc.2020. HaptX Gloves Development Kit. https://haptx.com/ accessed 15 September 2020.
[44]
Adam Jenkins, Pieris Kalligeros, Kami Vaniea, and Maria K Wolters. 2020. “Anyone Else Seeing this Error?”: Community, System Administrators, and Patch Information. (2020), 15.
[45]
Maurits Kaptein and Judy Robertson. 2012. Rethinking Statistical Analysis Methods for CHI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1105–1114. https://doi.org/10.1145/2207676.2208557
[46]
Christina Katsini, Yasmeen Abdrabou, George E. Raptis, Mohamed Khamis, and Florian Alt. 2020. The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–21. https://doi.org/10.1145/3313831.3376840
[47]
Mohamed Khamis, Florian Alt, Mariam Hassib, Emanuel von Zezschwitz, Regina Hasholzner, and Andreas Bulling. 2016. GazeTouchPass: Multimodal Authentication Using Gaze and Touch on Mobile Devices. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA) (CHI EA ’16). ACM, New York, NY, USA, 6. https://doi.org/10.1145/2851581.2892314
[48]
Mohamed Khamis, Linda Bandelow, Stina Schick, Dario Casadevall, Andreas Bulling, and Florian Alt. 2017. They are all after you: Investigating the Viability of a Threat Model that involves Multiple Shoulder Surfers. In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia (Stuttgart, Germany) (MUM ’17). ACM, New York, NY, USA, 5. https://doi.org/10.1145/3152832.3152851
[49]
Mohamed Khamis, Malin Eiband, Martin Zürn, and Heinrich Hussmann. 2018. EyeSpot: Leveraging Gaze to Protect Private Text Content on Mobile Devices from Shoulder Surfing. Multimodal Technologies and Interaction 2, 3, Article 45(2018), 15 pages. https://doi.org/10.3390/mti2030045
[50]
Mohamed Khamis, Mariam Hassib, Emanuel von Zezschwitz, Andreas Bulling, and Florian Alt. 2017. GazeTouchPIN: Protecting Sensitive Data on Mobile Devices using Secure Multimodal Authentication. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, Scotland) (ICMI 2017). ACM, New York, NY, USA, 5. https://doi.org/10.1145/3136755.3136809
[51]
Mohamed Khamis, Axel Hoesl, Alexander Klimczak, Martin Reiss, Florian Alt, and Andreas Bulling. 2017. EyeScout: Active Eye Tracking for Position and Movement Independent Gaze Interaction with Large Public Displays. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 155–166. https://doi.org/10.1145/3126594.3126630
[52]
Mohamed Khamis, Ludwig Trotter, Ville Mäkelä, Emanuel von Zezschwitz, Jens Le, Andreas Bulling, and Florian Alt. 2018. CueAuth: Comparing Touch, Mid-Air Gestures, and Gaze for Cue-Based Authentication on Situated Displays. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 174 (Dec. 2018), 22 pages. https://doi.org/10.1145/3287052
[53]
David Kim, Paul Dunphy, Pam Briggs, Jonathan Hook, John W. Nicholson, James Nicholson, and Patrick Olivier. 2010. Multi-Touch Authentication on Tabletops. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 1093–1102. https://doi.org/10.1145/1753326.1753489
[54]
Yaesol Kim, Hyun Jung Kim, and Young J. Kim. 2018. Encountered-type haptic display for large VR environment using per-plane reachability maps. Computer Animation and Virtual Worlds 29, 3-4 (2018), 11. https://doi.org/10.1002/cav.1814
[55]
Jesper Kjeldskov and Mikael B. Skov. 2014. Was It Worth the Hassle? Ten Years of Mobile HCI Research Discussions on Lab and Field Evaluations. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices and Services (Toronto, ON, Canada) (MobileHCI ’14). Association for Computing Machinery, New York, NY, USA, 43–52. https://doi.org/10.1145/2628363.2628398
[56]
Jesper Kjeldskov, Mikael B. Skov, Benedikte S. Als, and Rune T. Høegh. 2004. Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. In Mobile Human-Computer Interaction - MobileHCI 2004, Stephen Brewster and Mark Dunlop (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 61–73.
[57]
Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3173574.3173919
[58]
Lisa Koeman. 2020. HCI/UX Research: What methods do we use?https://lisakoeman.nl/blog/hci-ux-research-what-methods-do-we-use/ accessed 15 September 2020.
[59]
Katharina Krombholz, Heidelinde Hobel, Markus Huber, and Edgar Weippl. 2015. Advanced social engineering attacks. Journal of Information Security and Applications 22 (2015), 113 – 122. https://doi.org/10.1016/j.jisa.2014.09.005 Special Issue on Security of Information and Networks.
[60]
Katharina Krombholz, Thomas Hupperich, and Thorsten Holz. 2016. Use the Force: Evaluating Force-Sensitive Authentication for Mobile Devices. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016). USENIX Association, Denver, CO, 207–219. https://www.usenix.org/conference/soups2016/technical-sessions/presentation/krombholz
[61]
Manu Kumar, Tal Garfinkel, Dan Boneh, and Terry Winograd. 2007. Reducing Shoulder-Surfing by Using Gaze-Based Password Entry. In Proceedings of the 3rd Symposium on Usable Privacy and Security (Pittsburgh, Pennsylvania, USA) (SOUPS ’07). Association for Computing Machinery, New York, NY, USA, 13–19. https://doi.org/10.1145/1280680.1280683
[62]
Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. 707–710.
[63]
Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Providing Haptics to Walls and Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1471–1482. https://doi.org/10.1145/3025453.3025600
[64]
Ville Mäkelä, Sheikh Radiah Rahim Rivu, Saleh Alsherif, Mohamed Khamis, Chong Xiao, Lisa Marianne Borchert, Albrecht Schmidt, and Florian Alt. 2020. Virtual Field Studies: Conducting Studies on Public Displays in Virtual Reality. In Proceedings of the 38th Annual ACM Conference on Human Factors in Computing Systems (Honolulu, Hawaii, USA) (CHI ’20). ACM, New York, NY, USA, 10. https://doi.org/10.1145/3313831.3376796
[65]
Karola Marky, Verena Zimmermann, Alina Stöver, Philipp Hoffmann, Kai Kunze, and Max Mühlhäuser. 2020. All in One! User Perceptions on Centralized IoT Privacy Settings. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3383016
[66]
Diogo Marques, Tiago Guerreiro, Luis Carriço, Ivan Beschastnikh, and Konstantin Beznosov. 2019. Vulnerability & Blame: Making Sense of Unauthorized Access to Smartphones. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300819
[67]
Florian Mathis, John Williamson, Kami Vaniea, and Mohamed Khamis. 2020. RubikAuth: Fast and Secure Authentication in Virtual Reality. In Proceedings of the 38th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems(Honolulu, Hawaii, USA) (CHI EA ’20). ACM, New York, NY, USA, 9. https://doi.org/10.1145/3334480.3382827
[68]
Florian Mathis, John Williamson, Kami Vaniea, and Mohamed Khamis. 2021. Fast and Secure Authentication in Virtual Reality using Coordinated 3D Manipulation and Pointing. ACM Transactions on Computer-Human Interaction (ToCHI) 28, 1 (Jan. 2021), 18. https://doi.org/10.1145/3428121
[69]
A. M. Matwyshyn, A. Cui, A. D. Keromytis, and S. J. Stolfo. 2010. Ethics in security vulnerability research. IEEE Security Privacy 8, 2 (2010), 67–72. https://doi.org/10.1109/MSP.2010.67
[70]
Michelle L. Mazurek, Saranga Komanduri, Timothy Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Patrick Gage Kelley, Richard Shay, and Blase Ur. 2013. Measuring Password Guessability for an Entire University. In Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security (Berlin, Germany) (CCS ’13). Association for Computing Machinery, New York, NY, USA, 173–186. https://doi.org/10.1145/2508859.2516726
[71]
Lukas Mecke, Ken Pfeuffer, Sarah Prange, and Florian Alt. 2018. Open Sesame! User Perception of Physical, Biometric, and Behavioural Authentication Concepts to Open Doors. In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia (Cairo, Egypt) (MUM 2018). Association for Computing Machinery, New York, NY, USA, 153–159. https://doi.org/10.1145/3282894.3282923
[72]
Leap Motion. 2019. Unity Assets for Leap Motion Orion Beta. https://developer.leapmotion.com/unity accessed 26 August 2020.
[73]
Mehdi Moussaïd, Mubbasir Kapadia, Tyler Thrash, Robert W Sumner, Markus Gross, Dirk Helbing, and Christoph Hölscher. 2016. Crowd behaviour during high-stress evacuations in an immersive virtual environment. Journal of The Royal Society Interface 13, 122 (2016), 20160414. https://doi.org/10.1098/rsif.2016.0414
[74]
Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking Glass: A Field Study on Noticing Interactivity of a Shop Window. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 297–306. https://doi.org/10.1145/2207676.2207718
[75]
Evangelos Niforatos, Adam Palma, Roman Gluszny, Athanasios Vourvopoulos, and Fotis Liarokapis. 2020. Would You Do It?: Enacting Moral Dilemmas in Virtual Reality for Understanding Ethical Decision-Making. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376788
[76]
Borna Noureddin, Peter D Lawrence, and CF Man. 2005. A non-contact device for tracking gaze in a human computer interface. Computer Vision and Image Understanding 98, 1 (2005), 52–82. https://doi.org/10.1016/j.cviu.2004.07.005
[77]
World Health Organization. 2020. Coronavirus disease (COVID-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 accessed 15 September 2020.
[78]
Eyal Peer, Joachim Vosgerau, and Alessandro Acquisti. 2014. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior research methods 46, 4 (2014), 1023–1031. https://doi.org/10.3758/s13428-013-0434-y
[79]
Ken Pfeuffer, Melodie Vidal, Jayson Turner, Andreas Bulling, and Hans Gellersen. 2013. Pursuit Calibration: Making Gaze Calibration Less Tedious and More Flexible. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13). Association for Computing Machinery, New York, NY, USA, 261–270. https://doi.org/10.1145/2501988.2501998
[80]
Prolific. 2020. Quickly find research participants you can trust.https://www.prolific.co/ accessed 15 September 2020.
[81]
Susanne Putze, Dmitry Alexandrovsky, Felix Putze, Sebastian Höffner, Jan David Smeddinck, and Rainer Malaka. 2020. Breaking The Experience: Effects of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376144
[82]
Qualtrics. 2005. Qualtrics Experience Managements. https://www.qualtrics.com accessed 15 September 2020.
[83]
Razer. 2020. Razer Blade 15: NVIDIA GeForce RTX 2080. https://www.razer.com/gb-en/gaming-laptops/razer-blade accessed 15 September 2020.
[84]
Elissa M Redmiles, Yasemin Acar, Sascha Fahl, and Michelle L Mazurek. 2017. A summary of survey methodology best practices for security and privacy researchers. Technical Report. https://doi.org/10.13016/M22K2W
[85]
Elissa M. Redmiles, Ziyun Zhu, Sean Kross, Dhruv Kuchhal, Tudor Dumitras, and Michelle L. Mazurek. 2018. Asking for a Friend: Evaluating Response Biases in Security User Studies. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (Toronto, Canada) (CCS ’18). Association for Computing Machinery, New York, NY, USA, 1238–1255. https://doi.org/10.1145/3243734.3243740
[86]
Robert Rosenthal and Ralph L Rosnow. 2009. Artifacts in behavioral research. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195385540.001.0001
[87]
Alireza Sahami Shirazi, Peyman Moghadam, Hamed Ketabdar, and Albrecht Schmidt. 2012. Assessing the Vulnerability of Magnetic Gestural Authentication to Video-Based Shoulder Surfing Attacks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 2045–2048. https://doi.org/10.1145/2207676.2208352
[88]
Johnny Saldaña. 2015. The coding manual for qualitative researchers. Sage.
[89]
Gian-Luca Savino, Niklas Emanuel, Steven Kowalzik, Felix Kroll, Marvin C. Lange, Matthis Laudan, Rieke Leder, Zhanhua Liang, Dayana Markhabayeva, Martin Schmeißer, Nicolai Schütz, Carolin Stellmacher, Zihe Xu, Kerstin Bub, Thorsten Kluss, Jaime Maldonado, Ernst Kruijff, and Johannes Schöning. 2019. Comparing Pedestrian Navigation Methods in Virtual Reality and Real Life. In 2019 International Conference on Multimodal Interaction (Suzhou, China) (ICMI ’19). Association for Computing Machinery, New York, NY, USA, 16–25. https://doi.org/10.1145/3340555.3353741
[90]
Stefan Schmidt. 2009. Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences. Review of General Psychology 13, 2 (2009), 90–100. https://doi.org/10.1037/a0015108
[91]
Helmut Schrom-Feiertag, Volker Settgast, and Stefan Seer. 2017. Evaluation of indoor guidance systems using eye tracking in an immersive virtual environment. Spatial Cognition & Computation 17, 1-2 (2017), 163–183. https://doi.org/10.1080/13875868.2016.1228654
[92]
Ben Shneiderman, Catherine Plaisant, Maxine Cohen, Steven Jacobs, Niklas Elmqvist, and Nicholas Diakopoulos. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction (6th ed.). Pearson.
[93]
Vijay Sivaraman, Hassan Habibi Gharakheili, Clinton Fernandes, Narelle Clark, and Tanya Karliychuk. 2018. Smart IoT Devices in the Home: Security and Privacy Implications. IEEE Technology and Society Magazine 37, 2 (2018), 71–79. https://doi.org/10.1109/MTS.2018.2826079
[94]
Yunpeng Song, Yun Huang, Zhongmin Cai, and Jason I. Hong. 2020. I’m All Eyes and Ears: Exploring Effective Locators for Privacy Awareness in IoT Scenarios. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376585
[95]
Anthony Steed, Francisco Ortega, Adam Williams, Ernst Kruijff, Wolfgang Stuerzlinger, Anil Batmaz, Andrea Won, Evan Rosenberg, Adalberto Simeone, and Aleshia Hayes. 2020. Evaluating Immersive Experiences During COVID-19 and Beyond. Interactions (2020), 62–67. https://interactions.acm.org/blog/view/evaluating-immersive-experiences-during-covid-19-and-beyond
[96]
William G. Stillwell, David A. Seaver, and Ward Edwards. 1981. A comparison of weight approximation techniques in multiattribute utility decision making. Organizational Behavior and Human Performance 28, 1(1981), 62 – 77. https://doi.org/10.1016/0030-5073(81)90015-5
[97]
Hilary Sutcliffe. 2011. A report on responsible research and innovation. MATTER and the European Commission(2011), 34.
[98]
Mohammad Tahaei, Kami Vaniea, and Naomi Saphra. 2020. Understanding Privacy-Related Questions on Stack Overflow. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376768
[99]
Unity Technologies. 2020. Unity Developer Documentation: Start bringing your vision to life today with the Unity real-time 3D development platform.https://docs.unity3d.com/ScriptReference/Collider.html accessed 15 September 2020.
[100]
Tobii Technology. 2020. Tobii HTC VIVE Devkit. https://vr.tobii.com/sdk/develop/unity/getting-started/tobii-htc-dev-kit/ accessed 26 August 2020.
[101]
Lauren Thevin, Carine Briant, and Anke M. Brock. 2020. X-Road: Virtual Reality Glasses for Orientation and Mobility Training of People with Visual Impairments. ACM Trans. Access. Comput. 13, 2, Article 7 (April 2020), 47 pages. https://doi.org/10.1145/3377879
[102]
Tobii. 2020. Tobii Pro VR Integration. https://www.tobiipro.com/product-listing/vr-integration/ accessed 26 August 2020.
[103]
Mélodie Vidal, Andreas Bulling, and Hans Gellersen. 2013. Pursuits: Spontaneous Interaction with Displays Based on Smooth Pursuit Eye Movement and Moving Targets. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Zurich, Switzerland) (UbiComp ’13). Association for Computing Machinery, New York, NY, USA, 439–448. https://doi.org/10.1145/2493432.2493477
[104]
Alexandra Voit, Sven Mayer, Valentin Schwind, and Niels Henze. 2019. Online, VR, AR, Lab, and In-Situ: Comparison of Research Methods to Evaluate Smart Artifacts. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, Article 507, 12 pages. https://doi.org/10.1145/3290605.3300737
[105]
Emanuel von Zezschwitz, Alexander De Luca, Bruno Brunkow, and Heinrich Hussmann. 2015. SwiPIN: Fast and Secure PIN-Entry on Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 1403–1406. https://doi.org/10.1145/2702123.2702212
[106]
Emanuel von Zezschwitz, Paul Dunphy, and Alexander De Luca. 2013. Patterns in the Wild: A Field Study of the Usability of Pattern and Pin-Based Authentication on Mobile Devices. In Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services (Munich, Germany) (MobileHCI ’13). Association for Computing Machinery, New York, NY, USA, 261–270. https://doi.org/10.1145/2493190.2493231
[107]
Roman Weiss and Alexander De Luca. 2008. PassShapes: Utilizing Stroke Based Authentication to Increase Password Memorability. In Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges (Lund, Sweden) (NordiCHI ’08). Association for Computing Machinery, New York, NY, USA, 383–392. https://doi.org/10.1145/1463160.1463202
[108]
Oliver Wiese and Volker Roth. 2016. See You next Time: A Model for Modern Shoulder Surfers. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services(Florence, Italy) (MobileHCI ’16). Association for Computing Machinery, New York, NY, USA, 453–464. https://doi.org/10.1145/2935334.2935388
[109]
Max Wilson, Wendy Mackay, Ed Chi, Michael Bernstein, and Jeffrey Nichols. 2012. RepliCHI SIG: From a Panel to a New Submission Venue for Replication. In CHI ’12 Extended Abstracts on Human Factors in Computing Systems (Austin, Texas, USA) (CHI EA ’12). Association for Computing Machinery, New York, NY, USA, 1185–1188. https://doi.org/10.1145/2212776.2212419
[110]
Max L. Wilson, Ed H. Chi, Stuart Reeves, and David Coyle. 2014. RepliCHI: The Workshop II. In CHI ’14 Extended Abstracts on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI EA ’14). Association for Computing Machinery, New York, NY, USA, 33–36. https://doi.org/10.1145/2559206.2559233
[111]
Christian Winkler, Jan Gugenheimer, Alexander De Luca, Gabriel Haas, Philipp Speidel, David Dobbelstein, and Enrico Rukzio. 2015. Glass Unlock: Enhancing Security of Smartphone Unlocking through Leveraging a Private Near-Eye Display. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 1407–1410. https://doi.org/10.1145/2702123.2702316
[112]
Jacob O. Wobbrock and Julie A. Kientz. 2016. Research Contributions in Human-Computer Interaction. Interactions 23, 3 (April 2016), 38–44. https://doi.org/10.1145/2907069
[113]
Serena Zheng, Noah Apthorpe, Marshini Chetty, and Nick Feamster. 2018. User Perceptions of Smart Home IoT Privacy. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 200 (Nov. 2018), 20 pages. https://doi.org/10.1145/3274469

Cited By

View all
  • (2024)Recent Trends of Authentication Methods in Extended Reality: A SurveyApplied System Innovation10.3390/asi70300457:3(45)Online publication date: 28-May-2024
  • (2024)Simulating projective Augmented Reality Visualizations in Virtual Reality: Is VR a feasible Environment for medical AR Evaluations?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650843(1-8)Online publication date: 11-May-2024
  • (2024)AdaptiveVoice: Cognitively Adaptive Voice Interface for Driving AssistanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642876(1-18)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
May 2021
10862 pages
ISBN:9781450380966
DOI:10.1145/3411764
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Authentication
  2. Research Method
  3. Usable Security
  4. Virtual Reality

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

CHI '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)134
  • Downloads (Last 6 weeks)15
Reflects downloads up to 26 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Recent Trends of Authentication Methods in Extended Reality: A SurveyApplied System Innovation10.3390/asi70300457:3(45)Online publication date: 28-May-2024
  • (2024)Simulating projective Augmented Reality Visualizations in Virtual Reality: Is VR a feasible Environment for medical AR Evaluations?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650843(1-8)Online publication date: 11-May-2024
  • (2024)AdaptiveVoice: Cognitively Adaptive Voice Interface for Driving AssistanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642876(1-18)Online publication date: 11-May-2024
  • (2024)Evaluating Transferable Emotion Expressions for Zoomorphic Social Robots using VR Prototyping2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR62088.2024.00125(1087-1096)Online publication date: 21-Oct-2024
  • (2024)Enhancing Replicability in XR HCI Studies: A Survey-Based Approach2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00020(42-46)Online publication date: 21-Oct-2024
  • (2024)Co-existing with Drones: A Virtual Exploration of Proxemic Behaviours and Users’ Insights on Social DronesInternational Journal of Social Robotics10.1007/s12369-024-01111-716:3(547-567)Online publication date: 6-Mar-2024
  • (2024)Security and privacy in virtual reality: a literature surveyVirtual Reality10.1007/s10055-024-01079-929:1Online publication date: 20-Dec-2024
  • (2023)Emotional Experience in Human–Robot Collaboration: Suitability of Virtual Reality Scenarios to Study Interactions beyond Safety RestrictionsRobotics10.3390/robotics1206016812:6(168)Online publication date: 8-Dec-2023
  • (2023)Similarities and Differences between Immersive Virtual Reality, Real World, and Computer Screens: A Systematic Scoping Review in Human Behavior StudiesMultimodal Technologies and Interaction10.3390/mti70600567:6(56)Online publication date: 27-May-2023
  • (2023)Reality Anchors: Bringing Cues from Reality to Increase Acceptance of Immersive Technologies in TransitProceedings of the ACM on Human-Computer Interaction10.1145/36042667:MHCI(1-28)Online publication date: 13-Sep-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media