Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3544548.3581467acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Investigating Eyes-away Mid-air Typing in Virtual Reality using Squeeze haptics-based Postural Reinforcement

Published: 19 April 2023 Publication History

Abstract

In this paper, we investigate postural reinforcement haptics for mid-air typing using squeeze actuation on the wrist. We propose and validate eye-tracking based objective metrics that capture the impact of haptics on the user’s experience, which traditional performance metrics like speed and accuracy are not able to capture. To this end, we design four wrist-based haptic feedback conditions: no haptics, vibrations on keypress, squeeze+vibrations on keypress, and squeeze posture reinforcement + vibrations on keypress. We conduct a text input study with 48 participants to compare the four conditions on typing and gaze metrics. Our results show that for expert qwerty users, posture reinforcement haptics significantly benefit typing by reducing the visual attention on the keyboard by up to 44% relative to no haptics, thus enabling eyes-away behaviors.

Supplementary Material

MP4 File (3544548.3581467-video-preview.mp4)
Video Preview
MP4 File (3544548.3581467-video-figure.mp4)
Video Figure

References

[1]
Adilzhan Adilkhanov, Matteo Rubagotti, and Zhanat Kappassov. 2022. Haptic Devices: Wearability-Based Taxonomy and Literature Review. IEEE Access (2022).
[2]
A. S. Arif and W. Stuerzlinger. 2009. Analysis of text entry performance metrics. In 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH). 100–105. https://doi.org/10.1109/TIC-STH.2009.5444533
[3]
Carlos Bermejo and Pan Hui. 2017. A survey on haptic technologies for mobile augmented reality. arXiv preprint arXiv:1709.00698(2017).
[4]
Sidney Bovet, Aidan Kehoe, Katie Crowley, Noirin Curran, Mario Gutierrez, Mathieu Meisser, Damien O Sullivan, and Thomas Rouvinez. 2018. Using Traditional Keyboards in VR: SteamVR Developer Kit and Pilot Game User Study. In 2018 IEEE Games, Entertainment, Media Conference (GEM). IEEE, 1–9. https://doi.org/10.1109/GEM.2018.8516449
[5]
Doug A. Bowman, Christopher J. Rhoton, and Marcio S. Pinho. 2002. Text Input Techniques for Immersive Virtual Environments: An Empirical Comparison. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, 26 (sep 2002), 2154–2158. https://doi.org/10.1177/154193120204602611
[6]
Marta G. Carcedo, Soon Hau Chua, Simon Perrault, Paweł Wozniak, Raj Joshi, Mohammad Obaid, Morten Fjeld, and Shengdong Zhao. 2016. HaptiColor: Interpolating Color Information as Haptic Feedback to Assist the Colorblind. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3572–3583. https://doi.org/10.1145/2858036.2858220
[7]
Hsiang-Yu Chen, Joseph Santos, Matthew Graves, Kwangtaek Kim, and Hong Z. Tan. 2008. Tactor Localization at the Wrist. In Haptics: Perception, Devices and Scenarios, Manuel Ferre (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 209–218.
[8]
Tafadzwa Joseph Dube and Ahmed Sabbir Arif. 2019. Text Entry in Virtual Reality: A Comprehensive Review of the Literature. Springer, Cham, 419–437. https://doi.org/10.1007/978-3-030-22643-5_33
[9]
J. Dudley, H. Benko, D. Wigdor, and P. O. Kristensson. 2019. Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 289–300. https://doi.org/10.1109/ISMAR.2019.00027
[10]
John J. Dudley, Keith Vertanen, and Per Ola Kristensson. 2018. Fast and Precise Touch-Based Text Entry for Head-Mounted Augmented Reality with Variable Occlusion. ACM Transactions on Computer-Human Interaction 25, 6 (dec 2018), 1–40. https://doi.org/10.1145/3232163
[11]
Conor R. Foy, John J. Dudley, Aakar Gupta, Hrvoje Benko, and Per Ola Kristensson. 2021. Understanding, Detecting and Mitigating the Effects of Coactivations in Ten-Finger Mid-Air Typing in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 287, 11 pages. https://doi.org/10.1145/3411764.3445671
[12]
Alexandra Frischen, Andrew P Bayliss, and Steven P Tipper. 2007. Gaze cueing of attention: visual attention, social cognition, and individual differences.Psychological bulletin 133, 4 (2007), 694.
[13]
Gabriel González, José P. Molina, Arturo S. García, Diego Martínez, and Pascual González. 2009. Evaluation of Text Input Techniques in Immersive Virtual Environments. In New Trends on Human-Computer Interaction. Springer London, London, 1–10. https://doi.org/10.1007/978-1-84882-352-5_11
[14]
Jens Grubert, Lukas Witzani, Eyal Ofek, Michel Pahud, Matthias Kranz, and Per Ola Kristensson. 2018. Effects of Hand Representations for Typing in Virtual Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 151–158. https://doi.org/10.1109/VR.2018.8446250
[15]
Jens Grubert, Lukas Witzani, Eyal Ofek, Michel Pahud, Matthias Kranz, and Per Ola Kristensson. 2018. Text Entry in Immersive Head-Mounted Display-Based Virtual Reality Using Standard Keyboards. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 159–166. https://doi.org/10.1109/VR.2018.8446059
[16]
Aakar Gupta, Antony Albert Raj Irudayaraj, and Ravin Balakrishnan. 2017. HapticClench: Investigating Squeeze Sensations Using Memory Alloys. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 109–117. https://doi.org/10.1145/3126594.3126598
[17]
Aakar Gupta, Thomas Pietrzak, Nicolas Roussel, and Ravin Balakrishnan. 2016. Direct Manipulation in Tactile Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3683–3693. https://doi.org/10.1145/2858036.2858161
[18]
Aakar Gupta, Majed Samad, Kenrick Kin, Per Ola Kristensson, and Hrvoje Benko. 2020. Investigating remote tactile feedback for mid-air text-entry in virtual reality. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 350–360.
[19]
HTC. 2019. Vive. https://www.vive.com
[20]
Haiyan Jiang, Dongdong Weng, Zhenliang Zhang, Feng Chen, Haiyan Jiang, Dongdong Weng, Zhenliang Zhang, and Feng Chen. 2019. HiFinger: One-Handed Text Entry Technique for Virtual Environments Based on Touches between Fingers. Sensors 19, 14 (jul 2019), 3063. https://doi.org/10.3390/s19143063
[21]
Sooyoung Kim and Gerard Jounghyun Kim. 2004. Using keyboards with head mounted displays. In Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry - VRCAI ’04. ACM Press, New York, New York, USA, 336. https://doi.org/10.1145/1044588.1044662
[22]
Youngwon R. Kim and Gerard J. Kim. 2016. HoVR-type: smartphone as a typing interface in VR using hovering. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology - VRST ’16. ACM Press, New York, New York, USA, 333–334. https://doi.org/10.1145/2993369.2996330
[23]
Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, New York, New York, USA, 1–9. https://doi.org/10.1145/3173574.3173919
[24]
Katherine Kuchenbecker, Jonathan Fiene, and Gunter Niemeyer. 2006. Improving contact realism through event-based haptic feedback. IEEE transactions on visualization and computer graphics 12, 2(2006), 219–230.
[25]
Jaebong Lee and Seungmoon Choi. 2010. Effects of haptic guidance and disturbance on motor learning: Potential advantage of haptic disturbance. In 2010 IEEE Haptics Symposium. IEEE, 335–342.
[26]
Minkyung Lee and Woontack Woo. 2003. ARKB: 3D vision-based Augmented Reality Keyboard. In International Conferece on Artificial Reality and Telexisitence. Tokyo, Japan, 54–57. http://www.vrsj.org/ic-at/ICAT2003/php/upload/153_camera_38251238b626ed52ab0344b67e112d3d.pdf
[27]
Jia-Wei Lin, Ping-Hsuan Han, Jiun-Yu Lee, Yang-Sheng Chen, Ting-Wei Chang, Kuan-Wen Chen, and Yi-Ping Hung. 2017. Visualizing the keyboard in virtual reality for enhancing immersive experience. In ACM SIGGRAPH 2017 Posters on - SIGGRAPH ’17. ACM Press, New York, New York, USA, 1–2. https://doi.org/10.1145/3102163.3102175
[28]
I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI ’03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI EA ’03). ACM, New York, NY, USA, 754–755. https://doi.org/10.1145/765891.765971
[29]
Anders Markussen, Mikkel R. Jakobsen, and Kasper Hornbæk. 2013. Selection-Based Mid-Air Text Entry on Large Displays. Springer, Berlin, Heidelberg, 401–418. https://doi.org/10.1007/978-3-642-40483-2_28
[30]
Michael Matscheko, Alois Ferscha, Andreas Riener, and Manuel Lehner. 2010. Tactor placement in wrist worn wearables. In International Symposium on Wearable Computers (ISWC) 2010. 1–8. https://doi.org/10.1109/ISWC.2010.5665867
[31]
Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15. ACM Press, New York, New York, USA, 2143–2152. https://doi.org/10.1145/2702123.2702382
[32]
Taihei Ogitani, Yoshitaka Arahori, Yusuke Shinyama, and Katsuhiko Gondow. 2018. Space Saving Text Input Method for Head Mounted Display with Virtual 12-key Keyboard. In 2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA). IEEE, 342–349. https://doi.org/10.1109/AINA.2018.00059
[33]
J. Perret and E. Vander Poorten. 2018. Touching Virtual Reality: A Review of Haptic Gloves. In ACTUATOR 2018; 16th International Conference on New Actuators. 1–5.
[34]
Evan Pezent, Priyanshu Agarwal, Jessica Hartcher-O’Brien, Nicholas Colonnese, and Marcia K. O’Malley. 2022. Design, Control, and Psychophysics of Tasbi: A Force-Controlled Multimodal Haptic Bracelet. IEEE Transactions on Robotics(2022), 1–17. https://doi.org/10.1109/TRO.2022.3164840
[35]
Evan Pezent, Brandon Cambio, and Marcia K. O’Malley. 2021. Syntacts: Open-Source Software and Hardware for Audio-Controlled Haptics. IEEE Transactions on Haptics 14, 1 (2021), 225–233. https://doi.org/10.1109/TOH.2020.3002696
[36]
E. Pezent, A. Israr, M. Samad, S. Robinson, P. Agarwal, H. Benko, and N. Colonnese. 2019. Tasbi: Multisensory Squeeze and Vibrotactile Wrist Haptics for Augmented and Virtual Reality. In 2019 IEEE World Haptics Conference (WHC). 1–6. https://doi.org/10.1109/WHC.2019.8816098
[37]
Henning Pohl, Peter Brandes, Hung Ngo Quang, and Michael Rohs. 2017. Squeezeback: Pneumatic Compression for Notifications. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5318–5330. https://doi.org/10.1145/3025453.3025526
[38]
Manuel Prätorius, Dimitar Valkov, Ulrich Burgbacher, and Klaus Hinrichs. 2014. DigiTap: an eyes-free VR/AR symbolic input device. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology - VRST ’14. ACM Press, New York, New York, USA, 9–18. https://doi.org/10.1145/2671015.2671029
[39]
Thomas R Reppert, Karolina M Lempert, Paul W Glimcher, and Reza Shadmehr. 2015. Modulation of saccade vigor during value-based decision making. Journal of Neuroscience 35, 46 (2015), 15369–15378.
[40]
Naveen Sendhilnathan, Debaleena Basu, Michael E Goldberg, Jeffrey D Schall, and Aditya Murthy. 2021. Neural correlates of goal-directed and non–goal-directed movements. Proceedings of the National Academy of Sciences 118, 6 (2021), e2006372118.
[41]
Naveen Sendhilnathan, Ting Zhang, Ben Lafreniere, Tovi Grossman, and Tanya R Jonker. 2022. Detecting Input Recognition Errors and User Errors using Gaze Dynamics in Virtual Reality. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–19.
[42]
Mike Sinclair, Eyal Ofek, Mar Gonzalez-Franco, and Christian Holz. 2019. CapstanCrunch: A Haptic VR Controller with User-Supplied Force Feedback. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 815–829. https://doi.org/10.1145/3332165.3347891
[43]
Jeongmin Son, Sunggeun Ahn, Sunbum Kim, and Geehyuk Lee. 2019. Improving Two-Thumb Touchpad Typing in Virtual Reality. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA ’19. ACM Press, New York, New York, USA, 1–6. https://doi.org/10.1145/3290607.3312926
[44]
Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-based Text Entry in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, New York, New York, USA, 1–13. https://doi.org/10.1145/3173574.3174221
[45]
Typing.com. 2022. Typing Test. https://www.typing.com/student/typing-test/1-minute
[46]
UploadVR. 2019. Oculus Claims Breakthrough in Hand-tracking Accuracy. https://www.roadtovr.com/oculus-claims-breakthrough-in-hand-tracking-accuracy/
[47]
Chien-Min Wu, Chih-Wen Hsu, Tzu-Kuei Lee, and Shana Smith. 2017. A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment. Virtual Reality 21, 1 (01 Mar 2017), 19–29. https://doi.org/10.1007/s10055-016-0296-6
[48]
Motonori Yamaguchi, Matthew JC Crump, and Gordon D Logan. 2013. Speed–accuracy trade-off in skilled typewriting: Decomposing the contributions of hierarchical control loops.Journal of Experimental Psychology: Human Perception and Performance 39, 3(2013), 678.
[49]
Naoki Yanagihara and Buntarou Shizuki. 2018. Cubic Keyboard for Virtual Reality. In Proceedings of the Symposium on Spatial User Interaction - SUI ’18. ACM Press, New York, New York, USA, 170–170. https://doi.org/10.1145/3267782.3274687
[50]
Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST ’15. ACM Press, New York, New York, USA, 539–548. https://doi.org/10.1145/2807442.2807504
[51]
Difeng Yu, Kaixuan Fan, Heng Zhang, Diego Monteiro, Wenge Xu, and Hai-Ning Liang. 2018. PizzaText: Text Entry for Virtual Reality Systems Using Dual Thumbsticks. IEEE Transactions on Visualization and Computer Graphics 24, 11 (nov 2018), 2927–2935. https://doi.org/10.1109/TVCG.2018.2868581
[52]
Mengjia Zhu, Amirhossein H. Memar, Aakar Gupta, Majed Samad, Priyanshu Agarwal, Yon Visell, Sean J. Keller, and Nicholas Colonnese. 2020. PneuSleeve: In-Fabric Multimodal Actuation and Sensing in a Soft, Compact, and Expressive Haptic Sleeve. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376333

Cited By

View all
  • (2024)Interactive Mediation Techniques for Error-Aware Gesture Input SystemsProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670964(1-12)Online publication date: 3-Jun-2024
  • (2024)SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676384(1-19)Online publication date: 13-Oct-2024
  • (2024)GEARS: Generalizable Multi-Purpose Embeddings for Gaze and Hand Data in VR InteractionsProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659551(279-289)Online publication date: 22-Jun-2024
  • Show More Cited By

Index Terms

  1. Investigating Eyes-away Mid-air Typing in Virtual Reality using Squeeze haptics-based Postural Reinforcement

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
        April 2023
        14911 pages
        ISBN:9781450394215
        DOI:10.1145/3544548
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 19 April 2023

        Permissions

        Request permissions for this article.

        Check for updates

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        CHI '23
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

        Upcoming Conference

        CHI 2025
        ACM CHI Conference on Human Factors in Computing Systems
        April 26 - May 1, 2025
        Yokohama , Japan

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)480
        • Downloads (Last 6 weeks)23
        Reflects downloads up to 10 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Interactive Mediation Techniques for Error-Aware Gesture Input SystemsProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670964(1-12)Online publication date: 3-Jun-2024
        • (2024)SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676384(1-19)Online publication date: 13-Oct-2024
        • (2024)GEARS: Generalizable Multi-Purpose Embeddings for Gaze and Hand Data in VR InteractionsProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659551(279-289)Online publication date: 22-Jun-2024
        • (2024)A Stereohaptics Accessory for Spatial Computing PlatformsHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76803-3_19(325-340)Online publication date: 6-Dec-2024

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        Full Text

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media