research-article

Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands

Authors:

Pascal Knierim,

Valentin Schwind,

Anna Maria Feit,

Florian Nieuwenhuizen,

Niels HenzeAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 345, Pages 1 - 9

https://doi.org/10.1145/3173574.3173919

Published: 21 April 2018 Publication History

Abstract

Entering text is one of the most common tasks when interacting with computing systems. Virtual Reality (VR) presents a challenge as neither the user's hands nor the physical input devices are directly visible. Hence, conventional desktop peripherals are very slow, imprecise, and cumbersome. We developed a apparatus that tracks the user's hands, and a physical keyboard, and visualize them in VR. In a text input study with 32 participants, we investigated the achievable text entry speed and the effect of hand representations and transparency on typing performance, workload, and presence. With our apparatus, experienced typists benefited from seeing their hands, and reach almost outside-VR performance. Inexperienced typists profited from semi-transparent hands, which enabled them to type just 5.6 WPM slower than with a regular desktop setup. We conclude that optimizing the visualization of hands in VR is important, especially for inexperienced typists, to enable a high typing performance.

Supplementary Material

suppl.mov (pn3110-file5.mp4)

Supplemental video

Download
11.55 MB

References

[1]

Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1968--1979.

Digital Library

[2]

Doug A. Bowman, Christopher J. Rhoton, and Marcio S. Pinho. 2002. Text Input Techniques for Immersive Virtual Environments: An Empirical Comparison. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, 26 (2002), 2154--2158.

[3]

Anna Maria Feit, Daryl Weir, and Antti Oulasvirta. 2016. How We Type: Movement Strategies and Performance in Everyday Typing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 4262--4273.

Digital Library

[4]

Gabriel González, José P. Molina, Arturo S. García, Diego Martínez, and Pascual González. 2009. Evaluation of Text Input Techniques in Immersive Virtual Environments. Springer London, London, 109--118.

[5]

Jan Gugenheimer, David Dobbelstein, Christian Winkler, Gabriel Haas, and Enrico Rukzio. 2016. FaceTouch: Enabling Touch Interaction in Display Fixed UIs for Mobile Virtual Reality. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 49--60.

Digital Library

[6]

Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage Publications, 904--908.

[7]

Youngwon R. Kim and Gerard J. Kim. 2016. HoVR-type: Smartphone As a Typing Interface in VR Using Hovering. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology (VRST '16). ACM, New York, NY, USA, 333--334.

Digital Library

[8]

Jia-Wei Lin, Ping-Hsuan Han, Jiun-Yu Lee, Yang-Sheng Chen, Ting-Wei Chang, Kuan-Wen Chen, and Yi-Ping Hung. 2017. Visualizing the Keyboard in Virtual Reality for Enhancing Immersive Experience. In ACM SIGGRAPH 2017 Posters (SIGGRAPH '17). ACM, New York, NY, USA, Article 35, 2 pages.

Digital Library

[9]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, New York, NY, USA, 754--755.

Digital Library

[10]

Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2143--2152.

Digital Library

[11]

Mark McGill, Alexander Ng, and Stephen Brewster. 2017. I Am The Passenger: How Visual Motion Cues Can Influence Sickness For In-Car VR. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5655--5668.

Digital Library

[12]

Valentin Schwind, Pascal Knierim, Lewis Chuang, and Niels Henze. 2017a. "Where's Pinky?": The Effects of a Reduced Number of Fingers in Virtual Reality. In Proceedings of the 2017 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (CHI PLAY Companion '17). ACM, New York, NY, USA.

Digital Library

[13]

Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017b. "These Are Not My Hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 1577--1582.

Digital Library

[14]

R. William Soukoreff and I. Scott MacKenzie. 2003. Metrics for Text Entry Research: An Evaluation of MSD and KSPC, and a New Unified Error Metric. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 113--120.

Digital Library

[15]

Ivan E. Sutherland. 1968. A Head-mounted Three Dimensional Display. In Proceedings of the December 9--11, 1968, Fall Joint Computer Conference, Part I (AFIPS '68 (Fall, part I)). ACM, New York, NY, USA, 757--764.

Digital Library

[16]

Bruce H. Thomas. 2012. A Survey of Visual, Mixed, and Augmented Reality Gaming. Comput. Entertain. 10, 1, Article 3 (Dec. 2012), 33 pages.

Digital Library

[17]

Etienne van Wyk and Ruth de Villiers. 2009. Virtual Reality Training Applications for the Mining Industry. In Proceedings of the 6th International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa (AFRIGRAPH '09). ACM, New York, NY, USA, 53--63.

Digital Library

[18]

J Walker, S Kuhl, and K Vertanen. 2016. Decoder-assisted typing using an HMD and a physical keyboard. In Extended Abstracts of the the ACM Conference on Human Factors in Computing Systems, CHI, Vol. 16.

[19]

James Walker, Bochao Li, Keith Vertanen, and Scott Kuhl. 2017. Efficient Typing on a Visually Occluded Physical Keyboard. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5457--5461.

Digital Library

[20]

Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoper. Virtual Environ. 7, 3 (June 1998), 225--240.

Digital Library

[21]

Chun Yu, Yizheng Gu, Zhican Yang, Xin Yi, Hengliang Luo, and Yuanchun Shi. 2017. Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4479--4488.

Digital Library

Cited By

Chu QChen CHu HWu XHan B(2024)iHand: Hand Recognition-Based Text Input Method for Wearable DevicesComputers10.3390/computers1303008013:3(80)Online publication date: 19-Mar-2024
https://doi.org/10.3390/computers13030080
Cho KCha S(2024)Immersive Computing: Perspectives on Spatial Design Strategies for Individual Virtual Reality Workspace PlatformsArchives of Design Research10.15187/adr.2024.07.37.3.10537:3(105-117)Online publication date: 31-Jul-2024
https://doi.org/10.15187/adr.2024.07.37.3.105
Chiossi FEl Khaoudi YOu CSidenmark LZaky AFeuchtner TMayer S(2024)Evaluating Typing Performance in Different Mixed Reality Manifestations using Physiological FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/36981428:ISS(377-406)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698142
Show More Cited By

Index Terms

Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands
1. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Opportunities and Challenges of Text Input in Portable Virtual Reality
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Text input in virtual reality is not widespread outside of labs, although being increasingly researched. Current setups require powerful components that are expensive or not portable, hence preventing effective in-the-wild use. Latest technological ...
Typing on Midair Virtual Keyboards: Exploring Visual Designs and Interaction Styles
Human-Computer Interaction – INTERACT 2021
Abstract
We investigate typing on a QWERTY keyboard rendered in virtual reality. Our system tracks users’ hands in the virtual environment via a Leap Motion mounted on the front of a head mounted display. This allows typing on an auto-correcting midair ...
Efficient Typing on a Visually Occluded Physical Keyboard
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

The rise of affordable head-mounted displays (HMDs) has raised questions about how to best design user interfaces for this technology. This paper focuses on the use of HMDs for home and office applications that require substantial text input. A physical ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

German Research Foundation
German Federal Ministry of Education and Research

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

146
Total Citations
View Citations
2,391
Total Downloads

Downloads (Last 12 months)331
Downloads (Last 6 weeks)40

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chu QChen CHu HWu XHan B(2024)iHand: Hand Recognition-Based Text Input Method for Wearable DevicesComputers10.3390/computers1303008013:3(80)Online publication date: 19-Mar-2024
https://doi.org/10.3390/computers13030080
Cho KCha S(2024)Immersive Computing: Perspectives on Spatial Design Strategies for Individual Virtual Reality Workspace PlatformsArchives of Design Research10.15187/adr.2024.07.37.3.10537:3(105-117)Online publication date: 31-Jul-2024
https://doi.org/10.15187/adr.2024.07.37.3.105
Chiossi FEl Khaoudi YOu CSidenmark LZaky AFeuchtner TMayer S(2024)Evaluating Typing Performance in Different Mixed Reality Manifestations using Physiological FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/36981428:ISS(377-406)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698142
Rasanji VKumarasinghe NPethangoda RKarunarathna VRodrigo R(2024)Virtual Reality Bank: A Novel Banking Experience in Immersive WorldProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691576(193-202)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3691573.3691576
Akhoroz MYildirim C(2024)Poke Typing: Effects of Hand-Tracking Input and Key Representation on Mid-Air Text Entry Performance in Virtual RealityProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685734(293-301)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685734
Zhou HAyesh TFan CSarsenbayeva ZWithana A(2024)CoplayingVR: Understanding User Experience in Shared Control in Virtual RealityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785088:3(1-25)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678508
Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Zhang HLiang XLei YChen YHe ZZhang YChen LLin HHan TTian F(2024)Understanding the Effects of Restraining Finger Coactivation in Mid-Air Typing: from a Neuromechanical PerspectiveProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676441(1-18)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676441
Zhou HSomarathne PPeirispulle TFan CSarsenbayeva ZWithana A(2024)PairPlayVR: Shared Hand Control for Virtual GamesProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653057(311-314)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3653057
Mo WDechant MMarquardt NAyobi ASingh AHolloway C(2024)Exploring the Design Space of Input Modalities for Working in Mixed Reality on Long-haul FlightsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661560(2267-2285)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661560
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents