Article

Eliciting Wrist and Finger Gestures to Guide Recognizer Design

Authors:

Keiko Katsuragawa,

Edward LankAuthors Info & Claims

GI'19: Proceedings of the 45th Graphics Interface Conference on Proceedings of Graphics Interface 2019

Article No.: 9, Pages 1 - 9

https://doi.org/10.20380/GI2019.09

Published: 03 June 2019 Publication History

Abstract

While hand gestures, i.e. movements of the fingers and wrist, are a low-effort input modality, sensing and recognition of these smallscale gestures is challenging. In particular, while many authors have explored varying designs of hardware to support hand gesture input, each systems recognize their own gesture set, rendering challenging comparisons between different capture and recognition systems. In this paper, we explore the design of hand and finger gesture input by conducting an elicitation study to understand the tradeoffs between hand, wrist, and arm gestures. Alongside this, to evaluate the overall potential of wrist-worn recognition, we explore the design of hardware to recognize gestures by contrasting an IMUonly recognizer with a simple low-cost wrist-flex sensor. We discuss the implications of our work both to the comparative evaluation of systems and to the design of enhanced hardware sensing.

References

[1]

R. Aigner, D. Wigdor, H. Benko, M. Haller, D. Lindbauer, A. Ion, S. Zhao, and J. Tzu Kwan Valino Koh. Understanding Mid-Air Hand Gestures: A Study of Human Preferences in Usage of Gesture Types for HCI. Technical report, 11 2012.

[2]

S. S. Arefin Shimon, C. Lutton, Z. Xu, S. Morrison-Smith, C. Boucher, and J. Ruiz. Exploring Non-touchscreen Gestures for Smartwatches. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '16, pp. 3822-3833. ACM, New York, NY, USA, 2016.

Digital Library

[3]

G. Bailly, J. Müller, M. Rohs, D. Wigdor, and S. Kratz. ShoeSense: A New Perspective on Gestural Interaction and Wearable Applications. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '12, pp. 1239-1248. ACM, New York, NY, USA, 2012.

Digital Library

[4]

E. Chan, T. Seyed, W. Stuerzlinger, X.-D. Yang, and F. Maurer. User Elicitation on Single-hand Microgestures. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '16, pp. 3403-3414. ACM, New York, NY, USA, 2016. 2858589

Digital Library

[5]

A. Dementyev and J. A. Paradiso. WristFlex: Low-power Gesture Input with Wrist-worn Pressure Sensors. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST '14, pp. 161-166. ACM, New York, NY, USA, 2014. 1145/2642918.2647396

Digital Library

[6]

H. Dong, A. Danesh, N. Figueroa, and A. E. Saddik. An Elicitation Study on Gesture Preferences and Memorability Toward a Practical Hand-Gesture Vocabulary for Smart Televisions. IEEE Access, 3:543- 555, 2015.

[7]

A. Ferrone, X. Jiang, L. Maiolo, A. Pecora, L. Colace, and C. Menon. A fabric-based wearable band for hand gesture recognition based on filament strain sensors: A preliminary investigation. In 2016 IEEE Healthcare Innovation Point-Of-Care Technologies Conference, HIPOCT, pp. 113-116, 11 2016.

[8]

R. Fukui, M. Watanabe, T. Gyota, M. Shimosaka, and T. Sato. Hand Shape Classification with a Wrist Contour Sensor: Development of a Prototype Device. In Proceedings of the 13th International Conference on Ubiquitous Computing, UbiComp '11, pp. 311-314. ACM, New York, NY, USA, 2011.

Digital Library

[9]

F. Haque, M. Nancel, and D. Vogel. Myopoint: Pointing and Clicking Using Forearm Mounted Electromyography and Inertial Motion Sensors. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pp. 3653-3656. ACM, 2015.

Digital Library

[10]

C. Harrison, D. Tan, and D. Morris. Skinput: Appropriating the Body As an Input Surface. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '10, pp. 453-462. ACM, New York, NY, USA, 2010.

Digital Library

[11]

L. Hespanhol, M. Tomitsch, K. Grace, A. Collins, and J. Kay. Investigating Intuitiveness and Effectiveness of Gestures for Free Spatial Interaction with Large Displays. In Proceedings of the 2012 International Symposium on Pervasive Displays, PerDis '12, pp. 6:1-6:6. ACM, New York, NY, USA, 2012.

Digital Library

[12]

J. D. Hincapié-Ramos, X. Guo, P. Moghadasian, and P. Irani. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-air Interactions. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '14, pp. 1063-1072. ACM, New York, NY, USA, 2014.

Digital Library

[13]

P.-G. Jung, G. Lim, S. Kim, and K. Kong. A Wearable Gesture Recognition Device for Detecting Muscular Activities Based on Air-Pressure Sensors. IEEE Transactions on Industrial Informatics, 11(2):485-494, 4 2015.

[14]

K. Katsuragawa, K. Pietroszek, J. R. Wallace, and E. Lank. Watchpoint: Freehand Pointing with a Smartwatch in a Ubiquitous Display Environment. In Proceedings of the International Working Conference on Advanced Visual Interfaces, AVI '16, pp. 128-135. ACM, 2016.

Digital Library

[15]

D. Kim, O. Hilliges, S. Izadi, A. D. Butler, J. Chen, I. Oikonomidis, and P. Olivier. Digits: Freehand 3D Interactions Anywhere Using a Wrist-worn Gloveless Sensor. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST '12, pp. 167-176. ACM, 2012.

Digital Library

[16]

G. Laput, R. Xiao, and C. Harrison. ViBand: High-Fidelity BioAcoustic Sensing Using Commodity Smartwatch Accelerometers. In Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology, UIST '16, pp. 321-333. ACM, New York, NY, USA, 2016.

Digital Library

[17]

M. Liu, M. Nancel, and D. Vogel. Gunslinger: Subtle Arms-down Midair Interaction. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology, UIST '15, pp. 63-71. ACM, 2015.

Digital Library

[18]

Z. Lu, X. Chen, Q. Li, X. Zhang, and P. Zhou. A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices. IEEE Transactions on Human-Machine Systems, 44(2):293-299, 4 2014.

[19]

J. McIntosh, C. McNeill, M. Fraser, F. Kerber, M. Löchtefeld, and A. Krüger. EMPress: Practical Hand Gesture Classification with WristMounted EMG and Pressure Sensing. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '16, pp. 2332-2342. ACM, New York, NY, USA, 2016. 2858093

Digital Library

[20]

M. R. Morris, A. Danielescu, S. Drucker, D. Fisher, B. Lee, m. c. schraefel, and J. O. Wobbrock. Reducing Legacy Bias in Gesture Elicitation Studies. interactions, 21(3):40-45, 5 2014. 2591689

[21]

P. Paudyal, J. Lee, A. Banerjee, and S. K. S. Gupta. DyFAV: Dynamic Feature Selection and Voting for Real-time Recognition of Fingerspelled Alphabet Using Wearables. In Proceedings of the 22nd International Conference on Intelligent User Interfaces, IUI '17, pp. 457-467. ACM, New York, NY, USA, 2017. 3025216

[22]

K. Pietroszek, A. Kuzminykh, J. R. Wallace, and E. Lank. Smartcasting: A Discount 3D Interaction Technique for Public Displays. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design, OzCHI '14, pp. 119-128. ACM, New York, NY, USA, 2014. .2686629

Digital Library

[23]

T. Piumsomboon, A. Clark, M. Billinghurst, and A. Cockburn. Userdefined Gestures for Augmented Reality. In CHI '13 Extended Abstracts on Human Factors in Computing Systems, CHI EA '13, pp. 955-960. ACM, New York, NY, USA, 2013. 2468527

[24]

Z. Ren, J. Meng, J. Yuan, and Z. Zhang. Robust Hand Gesture Recognition with Kinect Sensor. In Proceedings of the 19th ACM International Conference on Multimedia, MM '11, pp. 759-760. ACM, New York, NY, USA, 2011.

Digital Library

[25]

J. Rico and S. Brewster. Gestures All Around Us: User Differences in Social Acceptability Perceptions of Gesture Based Interfaces. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI '09, pp. 64:1-64:2. ACM, New York, NY, USA, 2009. 1613936

Digital Library

[26]

J. Rico and S. Brewster. Usable gestures for mobile interfaces: Evaluating social acceptability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, pp. 887-896. ACM, New York, NY, USA, 2010.

Digital Library

[27]

J. Ruiz, Y. Li, and E. Lank. User-defined motion gestures for mobile interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '11, pp. 197-206. ACM, 2011. 1145/1978942.1978971

Digital Library

[28]

J. Ruiz and D. Vogel. Soft-constraints to reduce legacy and performance bias to elicit whole-body gestures with low arm fatigue. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3347-3350. ACM, 2015.

Digital Library

[29]

T. S. Saponas, D. S. Tan, D. Morris, R. Balakrishnan, J. Turner, and J. A. Landay. Enabling Always-available Input with Muscle-computer Interfaces. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, UIST '09, pp. 167-176. ACM, New York, NY, USA, 2009.

Digital Library

[30]

S. Siddhpuria, K. Katsuragawa, J. R. Wallace, and E. Lank. Exploring At-Your-Side Gestural Interaction for Ubiquitous Environments. In Proceedings of the 2017 Conference on Designing Interactive Systems, DIS '17, pp. 1111-1122. ACM, New York, NY, USA, 2017. 1145/3064663.3064695

Digital Library

[31]

R. Tidwell, R. Akl, S. Akumalla, S. Karlaputi, D. Struble, and K. Kavi. Evaluating the Feasibility of EMG and Bend Sensors for Classifying Hand Gestures. In Proceedings of the International Conference on Multimedia and Human Computer Interaction, MHCI '13, pp. 1-8. ASET, Toronto, ON, Canada, 2013.

[32]

R.-D. Vatavu and J. O. Wobbrock. Formalizing Agreement Analysis for Elicitation Studies: New Measures, Significance Test, and Toolkit. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pp. 1325-1334. ACM, New York, NY, USA, 2015.

Digital Library

[33]

R.-D. Vatavu and I.-A. Zaiti. Leap Gestures for TV: Insights from an Elicitation Study. In Proceedings of the 2014 ACM International Conference on Interactive Experiences for TV and Online Video, TVX '14, pp. 131-138. ACM, New York, NY, USA, 2014. 2602299.2602316

Digital Library

[34]

D. Vogel and R. Balakrishnan. Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, UIST '05, pp. 33-42. ACM, 2005.

Digital Library

[35]

D. Way and J. Paradiso. A Usability User Study Concerning Free-Hand Microgesture and Wrist-Worn Sensors. In Proceedings of the 2014 11th International Conference on Wearable and Implantable Body Sensor Networks, BSN '14, pp. 138-142. IEEE Computer Society, Washington, DC, USA, 2014.

Digital Library

[36]

H. Wen, J. Ramos Rojas, and A. K. Dey. Serendipity: Finger Gesture Recognition Using an Off-the-Shelf Smartwatch. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '16, pp. 3847-3851. ACM, New York, NY, USA, 2016. 2858036.2858466

Digital Library

[37]

D. Wigdor and D. Wixon. Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1st ed., 2011.

[38]

J. O. Wobbrock, M. R. Morris, and A. D. Wilson. User-defined Gestures for Surface Computing. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI '09, pp. 1083-1092. ACM, 2009.

Digital Library

[39]

Y. Zhang and C. Harrison. Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology, UIST '15, pp. 167-173. ACM, New York, NY, USA, 2015.

Digital Library

[40]

Y. Zhang, R. Xiao, and C. Harrison. Advancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography. In Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology, UIST '16, pp. 843-850. ACM, New York, NY, USA, 2016.

Digital Library

Cited By

Andrei ABilius LVatavu R(2024)Take a Seat, Make a Gesture: Charting User Preferences for On-Chair and From-Chair Gesture InputProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642028(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642028
Villarreal-Narvaez SSluÿters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
Vatavu RUngurean O(2022)Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor ImpairmentsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501964(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501964

Recommendations

Making bare hand input more accurate
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems

To enable seamless interaction with both virtual and physical worlds, computers need to reliably distinguish intended input from other hand movements. Accurately classifying this input can reduce mode switches needed between interacting with computers ...
Design of a novel finger exoskeleton with a sliding six-bar joint mechanism
AH '15: Proceedings of the 6th Augmented Human International Conference

The objective of the paper is to propose a novel design of a finger exoskeleton. The merit of the work is that the proposed mechanism is expected to eliminate interference and translational force on a finger. The design consists of 3 identical joint ...
Teaching motion gestures via recognizer feedback
IUI '14: Proceedings of the 19th international conference on Intelligent User Interfaces

When using motion gestures, 3D movements of a mobile phone, as an input modality, one significant challenge is how to teach end users the movement parameters necessary to successfully issue a command. Is a simple video or image depicting movement of a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

GI'19: Proceedings of the 45th Graphics Interface Conference on Proceedings of Graphics Interface 2019

June 2019

226 pages

ISBN:9780994786845

Conference Chairs:
Andrea Tagliasacchi
University of Victoria
,
Robert J. Teather
Carleton University

Sponsors

The Canadian Human-Computer Communications Society / Société Canadienne du Dialogue Humaine Machine (CHCCS/SCDHM)
Queen's University Kingston: Queen's University Kingston

In-Cooperation

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Canadian Human-Computer Communications Society

Waterloo, Canada

Publication History

Published: 03 June 2019

Check for updates

Author Tags

Qualifiers

Article

Acceptance Rates

GI'19 Paper Acceptance Rate 26 of 50 submissions, 52%;

Overall Acceptance Rate 206 of 508 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Andrei ABilius LVatavu R(2024)Take a Seat, Make a Gesture: Charting User Preferences for On-Chair and From-Chair Gesture InputProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642028(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642028
Villarreal-Narvaez SSluÿters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
Vatavu RUngurean O(2022)Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor ImpairmentsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501964(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501964

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

Media

Figures

Other

Tables

View Table of Contents