research-article

ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures

Authors:

Dustin Freeman,

Meredith Ringel Morris,

Daniel WigdorAuthors Info & Claims

ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces

Pages 165 - 172

https://doi.org/10.1145/1731903.1731935

Published: 23 November 2009 Publication History

Abstract

We present ShadowGuides, a system for in-situ learning of multi-touch and whole-hand gestures on interactive surfaces. ShadowGuides provides on-demand assistance to the user by combining visualizations of the user's current hand posture as interpreted by the system (feedback) and available postures and completion paths necessary to finish the gesture (feedforward). Our experiment compared participants learning gestures with ShadowGuides to those learning with video-based instruction. We found that participants learning with ShadowGuides remembered more gestures and expressed significantly higher preference for the help system.

Supplementary Material

JPG File (188.jpg)

Download
6.61 KB

MPG File (188.mpg)

Download
44.35 MB

References

[1]

Bau, O. and Mackay, W. E. OctoPocus: a dynamic guide for learning gesture-based command sets. In Proc. of ACM UIST 2008. p. 37--46.

Digital Library

[2]

Baudel, T., & Beaudouin-Lafon, M. Charade: remote control of objects using free-hand gestures. Communications of the ACM, Volume 36 Issue 7. 1993. p. 28--35.

Digital Library

[3]

Bragdon, A., et al. GestureBar: improving the approachability of gesture-based interfaces. In Proc. of ACM CHI 2009. p. 2269--2278.

Digital Library

[4]

Brandl, P., et al. Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. In Proc. of ACM AVI 2008. p. 154--161.

Digital Library

[5]

Buxton, W. A Three-State Model of Graphical Input. In Proc. of IFIP INTERACT 1990. p. 449--456.

Digital Library

[6]

Cao, X., Wilson, A., Balakrishnan, R., Hinckley, K., and Hudson, S. ShapeTouch: Leveraging Contact Shape on Interactive Surfaces. In Proc. of IEEE TABLETOP 2008. p. 139--146.

[7]

Forsythe, William. Synchronous Objects. http://synchronousobjects.osu.edu/

[8]

Grossman, T., Dragicevic, P., and Balakrishnan, R. Strategies for accelerating on-line learning of hotkeys. In Proc. of ACM CHI 2007. p. 1591--1600.

Digital Library

[9]

Kurtenbach, G. & Buxton, W. Issues in combining marking and direct manipulation techniques. In Proc. of ACM UIST 1991. p. 137--144.

Digital Library

[10]

Guest, A. H., Labanotation, Theatre Art Book, New York, 2005. 4th Ed.

[11]

Moran, T. P., Chiu, P., and van Melle, W. Pen-based interaction techniques for organizing material on an electronic whiteboard. In Proc. of ACM UIST 1997, p. 45--54.

Digital Library

[12]

Morris, M. R., et al. Cooperative Gestures: Multi-User Gestural Interactions for Co-located Groupware. In Proc. of ACM CHI 2006. p. 1201--1210.

Digital Library

[13]

Schneiderman, B. Direct manipulation: a step beyond programming languages. IEEE Computer Volume 16 Issue 8 (August 1983), p. 57--69.

Digital Library

[14]

Shoemaker, G., Tang, A., and Booth, K. S. Shadow reaching: a new perspective on interaction for large displays. In Proc. of ACM UIST 2007. p. 53--56.

Digital Library

[15]

Tang, A., Neustaedter, C. & Greenberg, S. VideoArms: embodiments in mixed presence groupware. In Proc. of BCS HCI 2006. p 85--102.

[16]

Tapia, M. A. and Kurtenbach, G. Some design refinements and principles on the appearance and behavior of marking menus. In Proc. of UIST 1995. p. 189--195.

Digital Library

[17]

Tversky, B., Bauer Morrison, J. Animation: can it facilitate? Int. Journal of Human-Computer Studies Volume 57, 2002. p. 247--262.

Digital Library

[18]

Vanacken, D. et al. Ghosts in the Interface: Meta-user Interface Visualizations as Guides for Multi-touch Interaction. In Proc. of IEEE TABLETOP 2008. p. 81--84.

[19]

Vogel, D. and Balakrishnan, R. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In Proc. of ACM UIST 2004, 137--146.

Digital Library

[20]

Vogel, D., Cudmore, M., Casiez, G., Balakrishnan, R., and Keliher, L. Hand occlusion with tablet-sized direct pen input. In Proc. of ACM CHI 2009. p. 557--566.

Digital Library

[21]

Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proc. of ACM CHI 2009. p 1083--1092.

Digital Library

[22]

Wu, M. and Balakrishnan, R. Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In Proc. of ACM UIST 2003. p. 193--202.

Digital Library

[23]

Wu, M., Shen, C., Ryall, K., Forlines, C., and Balakrishnan, R. Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces. In Proc. of IEEE TABLETOP 2006. p. 183--190.

Digital Library

[24]

Zhao, S., Agrawala, M., and Hinckley, K. Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus. In Proc. of ACM CHI 2006. p. 1077--1086.

Digital Library

Cited By

Khurana AGlueck MChilana P(2024)Do I Just Tap My Headset?Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314517:4(1-28)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631451
Wang XLafreniere BZhao J(2024)Exploring Visualizations for Precisely Guiding Bare Hand Gestures in Virtual RealityProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642935(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642935
Kandel JDuppen CZhang QJiang HAngelopoulos ANeall AWagh PSzafir DFuchs HLewek MSzafir D(2024)PD-Insighter: A Visual Analytics System to Monitor Daily Actions for Parkinson's Disease TreatmentProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642215(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642215
Show More Cited By

Index Terms

ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms
      1. Graphical user interfaces

Recommendations

Interacting with large displays from a distance with vision-tracked multi-finger gestural input
SIGGRAPH '06: ACM SIGGRAPH 2006 Sketches

We explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of ...
Grabbing at an angle: menu selection for fabric interfaces
ISWC '16: Proceedings of the 2016 ACM International Symposium on Wearable Computers

This paper investigates the pinch angle as a menu selection technique for two-dimensional foldable textile controllers. Based on the principles of marking menus, the selection of a menu item is performed by grabbing a fold at a specific angle, while ...
Analysing mouse and pen flick gestures
CHINZ '02: Proceedings of the SIGCHI-NZ Symposium on Computer-Human Interaction

Gesture based interfaces promise to increase the efficiency of user input, particularly in mobile computing where standard input devices such as the mouse and keyboard are impractical. This paper describes an investigation into the low-level physical ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces

November 2009

240 pages

ISBN:9781605587332

DOI:10.1145/1731903

General Chairs:
Gerald Morrison
SMART Technologies, Canada
,
Sriram Subramanian
University of Bristol, UK
,
Program Chairs:
Sheelagh Carpendale
University of Calgary, Canada
,
Michael Haller
Upper Austria University of Applied Sciences, Austria
,
Stacey Scott
University of Waterloo, Canada

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITS'09

Sponsor:

SIGCHI

ITS'09: The ACM International Conference on Interactive Tabletops and Surfaces

November 23 - 25, 2009

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 119 of 418 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

120
Total Citations
View Citations
1,547
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Khurana AGlueck MChilana P(2024)Do I Just Tap My Headset?Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314517:4(1-28)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631451
Wang XLafreniere BZhao J(2024)Exploring Visualizations for Precisely Guiding Bare Hand Gestures in Virtual RealityProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642935(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642935
Kandel JDuppen CZhang QJiang HAngelopoulos ANeall AWagh PSzafir DFuchs HLewek MSzafir D(2024)PD-Insighter: A Visual Analytics System to Monitor Daily Actions for Parkinson's Disease TreatmentProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642215(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642215
Yu XLee BSedlmair M(2024)Design Space of Visual Feedforward And Corrective Feedback in XR-Based Motion Guidance SystemsProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642143(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642143
Rubab SZaman MRashid UYu LWu Y(2024)Audio-visual training and feedback to learn touch-based gesturesJournal of Visualization10.1007/s12650-024-01012-xOnline publication date: 17-Jun-2024
https://doi.org/10.1007/s12650-024-01012-x
Kim SHong S(2023)Effects of Footpad Slope, Movement Direction and Contact Part of Foot on Foot-Based InteractionsApplied Sciences10.3390/app1311663613:11(6636)Online publication date: 30-May-2023
https://doi.org/10.3390/app13116636
Muresan AMcintosh JHornbæk K(2023)Using Feedforward to Reveal Interaction Possibilities in Virtual RealityACM Transactions on Computer-Human Interaction10.1145/360362330:6(1-47)Online publication date: 25-Sep-2023
https://dl.acm.org/doi/10.1145/3603623
Sabnis NWittchen DVega GReed CStrohmeier P(2023)Tactile Symbols with Continuous and Motion-Coupled Vibration: An Exploration of using Embodied Experiences for Hermeneutic DesignProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581356(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581356
Nguyen BHan JHouben MBayoumi YVande Moere A(2023)Engaging Passers-by with Rhythm: Applying Feedforward Learning to a Xylophonic Media Architecture FacadeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580761(1-21)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580761
Goguey AOrtega M(2023)Studies and guidelines for two concurrent stroke gesturesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102942170:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.ijhcs.2022.102942
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents