abstract

Hand-free natural user interface for VR HMD with IR based facial gesture tracking sensor

Authors:

Jinhyuk Kim,

Jaekwang Cha,

Hojun Lee,

Shiho KimAuthors Info & Claims

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

Article No.: 62, Pages 1 - 2

https://doi.org/10.1145/3139131.3143420

Published: 08 November 2017 Publication History

Get Access

Abstract

We proposed a hand-free Natural User Interface (NUI) for VR Head-Mounted-Display (HMD) with infra-red (IR) based sensor tracking facial gestures. We have realized NUI based on the real-time recognition of user intuitions for VR HMD without any additional control devices except for a built-in Gyroscope and IR couplers with readout circuitry integrated in the foam interface of an HMD. We implemented seven control commands affordable for 3D interactions with virtual objects. The experimental data show that the proposed system provides a convenient and efficient 2D/3D user interface for both manipulating objects and controlling commands while wearing a VR HMD.

References

[1]

Doug A Bowman, Jian Chen, Chadwick A Wingrave, John F Lucas, Andrew Ray, Nicholas F Polys, Qing Li, Yonca Haciahmetoglu, Ji-Sun Kim, Seonho Kim, et al. 2006. New directions in 3D user interfaces. IJVR 5, 2 (2006), 3--14.

Crossref

Google Scholar

[2]

Doug A Bowman and Chadwick A Wingrave. 2001. Design and evaluation of menu systems for immersive virtual environments. In Virtual Reality, 2001. Proceedings. IEEE. IEEE, 149--156.

Digital Library

Google Scholar

[3]

Jaekwang Cha, Jinhyuk Kim, and Shiho Kim. 2016. An IR-based facial expression tracking sensor for head-mounted displays. In SENSORS, 2016 IEEE. IEEE, 1--3.

Google Scholar

[4]

Martin G Helander. 2014. Handbook of human-computer interaction. Elsevier.

Google Scholar

[5]

Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research methods in human-computer interaction. Morgan Kaufmann.

Google Scholar

[6]

Sangho Lee, Gyutae Ha, Jaekwang Cha, Jinhyeok Kim, Hojun Lee, and Shiho Kim. 2015. CyberTouch-Touch and Cursor Interface for VR HMD. In International Conference on Human-Computer Interaction. Springer, 503--507.

Crossref

Google Scholar

[7]

Weiyuan Liu. 2010. Natural user interface-next mainstream product user interface. In Computer-Aided Industrial Design & Conceptual Design (CAIDCD), 2010 IEEE 11th International Conference on, Vol. 1. IEEE, 203--205.

Google Scholar

Cited By

View all

Lai YSun MLi Z(2024)GazePuffer: Hands-Free Input Method Leveraging Puff Cheeks for VR2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00055(331-341)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00055
Aslam FZhao R(2024)Voice-Augmented Virtual Reality Interface for Serious Games2024 IEEE Conference on Games (CoG)10.1109/CoG60054.2024.10645616(1-8)Online publication date: 5-Aug-2024
https://doi.org/10.1109/CoG60054.2024.10645616
Liu XWang LKe WIm S(2024)Object manipulation based on the head manipulation space in VRInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103346(103346)Online publication date: Aug-2024
https://doi.org/10.1016/j.ijhcs.2024.103346
Show More Cited By

Index Terms

Hand-free natural user interface for VR HMD with IR based facial gesture tracking sensor
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
      2. Pointing

Recommendations

Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

The natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...
Navigating in VR using free-hand gestures and embodied controllers: A comparative evaluation
VRST '23: Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology

While natural body-based movements are essential features for immersive VR (Virtual Reality) experiences, most of the available input techniques for navigation in VR involve the use of hand-held controllers. Alternatively, while body-based input for VR ...
Infrared bundle adjusting and clustering method for head-mounted display and Leap Motion calibration

Leap Motion has become widely used due to its ability to recognize intuitive hand gestures or accurate finger positions. Attaching a Leap Motion to a virtual reality head-mounted display (VR HMD) is highly interoperable with virtual objects in virtual ...

Comments

Information & Contributors

Information

Published In

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

November 2017

437 pages

ISBN:9781450355483

DOI:10.1145/3139131

General Chairs:
Morten Fjeld
Chalmers University of Technology
,
Marco Fratarcangeli
Chalmers University of Technology
,
Daniel Sjölie
University of Gothenburg
,
Program Chairs:
Oliver Staadt
University of Rostock
,
Jonas Unger
Linköping University

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2017

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Ministry of Science and ICT

Conference

VRST '17

Sponsor:

VRST '17: 23rd ACM Symposium on Virtual Reality Software and Technology

November 8 - 10, 2017

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
481
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lai YSun MLi Z(2024)GazePuffer: Hands-Free Input Method Leveraging Puff Cheeks for VR2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00055(331-341)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00055
Aslam FZhao R(2024)Voice-Augmented Virtual Reality Interface for Serious Games2024 IEEE Conference on Games (CoG)10.1109/CoG60054.2024.10645616(1-8)Online publication date: 5-Aug-2024
https://doi.org/10.1109/CoG60054.2024.10645616
Liu XWang LKe WIm S(2024)Object manipulation based on the head manipulation space in VRInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103346(103346)Online publication date: Aug-2024
https://doi.org/10.1016/j.ijhcs.2024.103346
Hombeck JVoigt HHeggemann TDatta RLawonn K(2023)Tell Me Where To Go: Voice-Controlled Hands-Free Locomotion for Virtual Reality Systems2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR55154.2023.00028(123-134)Online publication date: Mar-2023
https://doi.org/10.1109/VR55154.2023.00028
Han YZhang XZhang NMeng SLiu TWang SPan MZhang XYi J(2022)Hybrid Target Selections by ”Hand Gestures + Facial Expression” for a Rehabilitation RobotSensors10.3390/s2301023723:1(237)Online publication date: 26-Dec-2022
https://doi.org/10.3390/s23010237
Shen XYan YYu CShi Y(2022)ClenchClickProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503276:3(1-26)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550327
Zhou XGuo YJia LJin YLi HXue C(2022)A study of button size for virtual hand interaction in virtual environments based on clicking performanceMultimedia Tools and Applications10.1007/s11042-022-14038-w82:10(15903-15918)Online publication date: 15-Oct-2022
https://doi.org/10.1007/s11042-022-14038-w
Nam HSeol KLee JCho HJung S(2021)Review of Capacitive Touchscreen Technologies: Overview, Research Trends, and Machine Learning ApproachesSensors10.3390/s2114477621:14(4776)Online publication date: 13-Jul-2021
https://doi.org/10.3390/s21144776
Monteiro PGoncalves GCoelho HMelo MBessa M(2021)Hands-free interaction in immersive virtual reality: A systematic reviewIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.306768727:5(2702-2713)Online publication date: May-2021
https://doi.org/10.1109/TVCG.2021.3067687
Kim JCha JKim S(2020)Hands-Free User Interface for VR Headsets Based on In Situ Facial Gesture SensingSensors10.3390/s2024720620:24(7206)Online publication date: 16-Dec-2020
https://doi.org/10.3390/s20247206
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Finger identification and hand gesture recognition techniques for natural user interface

Navigating in VR using free-hand gestures and embodied controllers: A comparative evaluation

Infrared bundle adjusting and clustering method for head-mounted display and Leap Motion calibration