research-article

A User-Defined Gesture Set for Music Interaction in Immersive Virtual Environment

Authors:

Noris Mohd Norowi,

Azrul Hazri JantanAuthors Info & Claims

CHIuXiD '17: Proceedings of the 3rd International Conference on Human-Computer Interaction and User Experience in Indonesia

Pages 44 - 51

https://doi.org/10.1145/3077343.3077348

Published: 18 April 2017 Publication History

Abstract

In recent years, hand-tracking technologies had been implemented in Virtual Reality application, allowing users to use natural hand gesture for interaction within the environment. However, little efforts have been conducted in understanding user's preference when they use their hands to interact with the VR world. In this paper, the result of a guessability test for hand gestures in order to operate musical tasks to support music interaction within immersive Virtual Environment. A total number of 750 gestures have been elicited from 15 participants for 50 selected tasks, including 10 musical tasks. Our result enables a smaller size of gesture set to be elicited from the users. The implications of this work can be relevant in hand gesture design, gesture interaction, and gestural interfaces design for music interaction, all of which are highlighted in this study.

References

[1]

J. O. Wobbrock, M. R. Morris and A. D. Wilson, "User-defined gestures for surface computing" Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1083--1092, 2009.

Digital Library

[2]

J. Ruiz, Y. Li and E. Lank, "User-defined motion gestures for mobile interaction" Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 197--206, 2011.

Digital Library

[3]

T. Piumsomboon, A. Clark and M. Billinghurst, "User-defined gestures for augmented reality" IFIP Conference on Human-Computer Interaction, pp. 282--299, 2013.

Digital Library

[4]

D. England, "Whole body interaction: An introduction" Whole Body Interaction, pp. 1--5, 2011.

[5]

S. Jordà, G. Geiger, M. Alonso and M. Kaltenbrunner, "The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces" Proceedings of the 1st international conference on Tangible and embedded interaction, pp. 139--146, 2007.

Digital Library

[6]

S. Holland, "Learning about harmony with Harmony Space: an overview" Music education: An artificial intelligence approach, pp. 24--40, 1994.

[7]

J. Bird, S. Holland, P. Marshall, Y. Rogers and A. Clark, "Feel the force: Using tactile technologies to investigate the extended mind" Proceedings of Devices that Alter Perception workshop (DAP 08), 2008.

[8]

A. Bouwer, S. Holland and M. Dalgleish, "Song walker harmony space: Embodied interaction design for complex musical skills" Music and human-computer interaction, pp. 207--221, 2013.

[9]

A. F. Blackwell, C. Britton, A. Cox, T. R. Green, C. Gurr, G. Kadoda, M. S. Kutar, M. Lomes, C. L. Nehaniv, M. Petre and C. Roast, "Cognitive dimensions of notations: Design tools for cognitive technology" Cognitive Technology: Instruments of Mind, pp. 325--341, 2001.

[10]

J. Nielsen, "Noncommand user interfaces" Communications of the ACM, vol. 36, no. 4, pp. 83--99, 1993.

Digital Library

[11]

Z. Feng, S. Xu, X. Zhang, L. Jin, Z. Ye and W. Yang, "Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system" in ACM, 2012.

[12]

N. Xu, W. Wang and X. Qu, "Recognition of in-air handwritten Chinese character based on Leap Motion Controller" International Conference on Image and Graphics, pp. 160--168, August 2015.

[13]

Y. Niu and H. Chen, "Gesture authentication with touch input for mobile devices" International Conference on Security and Privacy in Mobile Information and Communication Systems, pp. 13--24, 2011.

[14]

G. Ren and E. O'Neill, "Freehand gestural text entry for interactive TV" Proceedings of the 11th european conference on Interactive TV and video, pp. 121--130, 2013.

Digital Library

[15]

J. Cui, W. F. DIeter, A. Kuijper and A. Sourin, "Mid-Air Gestures for Virtual Modeling with Leap Motion" International Conference on Distributed, Ambient, and Pervasive Interactions, pp. 221--230, 2016.

[16]

S. Garbaya and U. Zaldivar-Colado, "The affect of contact force sensations on user performance in virtual assembly tasks" Virtual Reality, vol. 11, no. 4, pp. 287--299, 2007.

Digital Library

[17]

C. Lewis and R. Mack, "Learning to use a text processing system: Evidence from "thinking aloud" protocols" Proceedings of the 1982 conference on Human factors in computing systems, pp. 387--392, 1982.

Digital Library

Cited By

Villarreal-Narvaez SSluÿters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
Graichen LGraichen MNudzor J(2023)Integrating Mid-Air Gestures into an Extended Reality Application: A Methodological ApproachProceedings of the 2023 7th International Conference on Virtual and Augmented Reality Simulations10.1145/3603421.3603422(1-6)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3603421.3603422
Hosseini MIhmels TChen ZKoelle MMüller HBoll S(2023)Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across DomainsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581420(1-24)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581420
Show More Cited By

Index Terms

A User-Defined Gesture Set for Music Interaction in Immersive Virtual Environment
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

User-defined gestures for augmented reality
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Recently there has been an increase in research of hand gestures for interaction in the area of Augmented Reality (AR). However this research has focused on developer designed gestures, and little is known about user preference and behavior for gestures ...
User-defined gestures for surface computing
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Many surface computing prototypes have employed gestures created by system designers. Although such gestures are appropriate for early investigations, they are not necessarily reflective of user behavior. We present an approach to designing tabletop ...
User Defined Eye Movement-Based Interaction for Virtual Reality
Cross-Cultural Design. Methods, Tools, and Users
Abstract
Most of the applications of eye movement-based interaction in VR are limited to blinking and gaze at present, however, gaze gestures were neglected. Therefore, the potential of eye movement-based interaction in VR is far from being realized. In ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CHIuXiD '17: Proceedings of the 3rd International Conference on Human-Computer Interaction and User Experience in Indonesia

April 2017

76 pages

ISBN:9781450352123

DOI:10.1145/3077343

Editors:
Eunice Sari
UX Indonesia
,
Adi Basuki Tedjasaputra
UX Indonesia

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHIuXiD '17

CHIuXiD '17: 3rd International Conference in Human-Computer Interaction and User Experience in Indonesia

April 18 - 21, 2017

Jakarta, Indonesia

Acceptance Rates

Overall Acceptance Rate 22 of 50 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
439
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)3

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Villarreal-Narvaez SSluÿters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
Graichen LGraichen MNudzor J(2023)Integrating Mid-Air Gestures into an Extended Reality Application: A Methodological ApproachProceedings of the 2023 7th International Conference on Virtual and Augmented Reality Simulations10.1145/3603421.3603422(1-6)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3603421.3603422
Hosseini MIhmels TChen ZKoelle MMüller HBoll S(2023)Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across DomainsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581420(1-24)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581420
Graichen LGraichen M(2023)Do Users Tolerate Errors? Effects of Observed Failures on the Subjective Evaluation of a Gesture-Based Virtual Reality ApplicationComputer-Human Interaction Research and Applications10.1007/978-3-031-49425-3_14(232-243)Online publication date: 23-Dec-2023
https://doi.org/10.1007/978-3-031-49425-3_14
Zhou XWilliams AOrtega F(2022)Towards Establishing Consistent Proposal Binning Methods for Unimodal and Multimodal Interaction Elicitation StudiesHuman-Computer Interaction. Theoretical Approaches and Design Methods10.1007/978-3-031-05311-5_25(356-368)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-05311-5_25
Cihanská Machová A(2021)Experimentálne prístupy k práci so zvukom vo virtuálnej realiteArteActa10.62804/aa.2021.0024:5(53-84)Online publication date: 15-Jun-2021
https://doi.org/10.62804/aa.2021.002
Enriquez KPalacios MPallo DGuerrero G(2020)SENSE: Sensory component VR application for hearing impaired people to enhance the music experience2020 15th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI49556.2020.9140447(1-6)Online publication date: Jun-2020
https://doi.org/10.23919/CISTI49556.2020.9140447
Villarreal-Narvaez SVanderdonckt JVatavu RWobbrock JWakkary RAndersen KOdom WDesjardins APetersen M(2020)A Systematic Review of Gesture Elicitation StudiesProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395511(855-872)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395511
Wei XXi RHou W(2020)User-Centric AR Sceneized Gesture Interaction DesignVirtual, Augmented and Mixed Reality. Design and Interaction10.1007/978-3-030-49695-1_24(367-378)Online publication date: 10-Jul-2020
https://doi.org/10.1007/978-3-030-49695-1_24
Arora RKazi RKaufman DLi WSingh KGuimbretière FBernstein MReinecke K(2019)MagicalHandsProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology10.1145/3332165.3347942(463-477)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3332165.3347942
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten