note

How Memorizing Positions or Directions Affects Gesture Learning?

Authors:

Bruno Fruchard,

Eric Lecolinet,

Olivier ChapuisAuthors Info & Claims

ISS '18: Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces

Pages 107 - 114

https://doi.org/10.1145/3279778.3279787

Published: 19 November 2018 Publication History

Abstract

Various techniques have been proposed to faster command selection. Many of them either rely on directional gestures (e.g. Marking menus) or pointing gestures using a spatially-stable arrangement of items (e.g. FastTap). Both types of techniques are known to leverage memorization, but not necessarily for the same reasons. In this paper, we investigate whether using directions or positions affects gesture learning. Our study shows that, while recall rates are not significantly different, participants used the novice mode more often and spent more time while learning commands with directional gestures, and they also reported more physical and mental efforts. Moreover, this study also highlights the importance of semantic relationships between gestural commands and reports on the memorization strategies that were elaborated by the participants.

Supplementary Material

MP4 File (p107-fruchard.mp4)

Download
391.96 MB

References

[1]

Caroline Appert and Shumin Zhai. 2009. Using strokes as command shortcuts: Cognitive benefits and toolkit support. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, 2289--2298.

Digital Library

[2]

A.D. Baddeley. 2013. Essentials of Human Memory (Classic Edition). Psychology Press, East Sussex, UK.

[3]

Gilles Bailly, Eric Lecolinet, and Laurence Nigay. 2008. Flower Menus: A New Type of Marking Menu with Large Menu Breadth, Within Groups and Efficient Expert Mode Memorization. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '08). ACM, 15--22.

Digital Library

[4]

Gilles Bailly, Eric Lecolinet, and Laurence Nigay. 2016. Visual menu techniques. ACM Comput. Surv. 49, 4, Article 60, 41 pages.

Digital Library

[5]

Olivier Bau and Wendy E. Mackay. 2008. OctoPocus: A Dynamic Guide for Learning Gesture-based Command Sets. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08). ACM, 37--46.

Digital Library

[6]

Thomas Baudel and Michel Beaudouin-Lafon. 1993. Charade: Remote control of objects using free-hand gestures. Commun. ACM 36, 7, 28--35.

Digital Library

[7]

Joanna Bergstrom-Lehtovirta, Sebastian Boring, and Kasper Hornbæk. 2017. Placing and recalling virtual items on the skin. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, 1497--1507.

Digital Library

[8]

Marvin M Chun and Yuhong Jiang. 1998. Contextual cueing: Implicit learning and memory of visual context guides spatial attention. Cognitive psychology 36, 1, 28--71.

[9]

Christopher Clarke, Alessio Bellino, Augusto Esteves, and Hans Gellersen. 2017. Remote control by body movement in synchrony with orbiting widgets: an evaluation of tracematch. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3, Article 45, 23 pages.

Digital Library

[10]

Andy Cockburn, Per Ola Kristensson, Jason Alexander, and Shumin Zhai. 2007. Hard lessons: Effort-inducing interfaces benefit spatial learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, 1571--1580.

Digital Library

[11]

Christian F. Doeller and Neil Burgess. 2008. Distinct error-correcting and incidental learning of location relative to landmarks and boundaries. Proceedings of the National Academy of Sciences 105, 15, 5909--5914.

[12]

Bruno Fruchard, Eric Lecolinet, and Olivier Chapuis. 2017. MarkPad: Augmenting Touchpads for Command Selection. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, 5630--5642.

Digital Library

[13]

Bruno Fruchard, Eric Lecolinet, and Olivier Chapuis. 2018a. Impact of semantic aids on command memorization for on-body interaction and directional gestures. In Proceedings of the 2018 International Conference on Advanced Visual Interfaces (AVI '18). ACM, Article 14, 9 pages.

Digital Library

[14]

Bruno Fruchard, Eric Lecolinet, and Olivier Chapuis. 2018b. Mémorisation de commandes: Positions spatiales versus gestes directionnels. In Proceedings of the 30th international conference of the Association Francophone d'Interaction Homme-Machine (IHM '18). ACM, 8 pages. To appear.

[15]

Tovi Grossman, Pierre Dragicevic, and Ravin Balakrishnan. 2007. Strategies for accelerating on-line learning of hotkeys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, 1591--1600.

Digital Library

[16]

Carl Gutwin and Andy Cockburn. 2006. Improving List Revisitation with ListMaps . In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '06). ACM, 396--403.

Digital Library

[17]

Carl Gutwin, Andy Cockburn, and Benjamin Lafreniere. 2015. Testing the rehearsal hypothesis with two fasttap interfaces. In Proceedings of the 41st Graphics Interface Conference (GI '15). Canadian Information Processing Society, 223--231. http://dl.acm.org/citation.cfm?id=2788890.2788930

Digital Library

[18]

Carl Gutwin, Andy Cockburn, Joey Scarr, Sylvain Malacria, and Scott C. Olson. 2014. Faster Command Selection on Tablets with FastTap. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, 2617--2626.

Digital Library

[19]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139 -- 183.

[20]

Gordon Kurtenbach and William Buxton. 1993. The limits of expert performance using hierarchic marking menus. In Proceedings of the INTERCHI '93 Conference on Human Factors in Computing Systems (INTERCHI '93). IOS Press, 482--487. http://dl.acm.org/citation.cfm?id=164632.164977

Digital Library

[21]

Jean M Mandler, Dale Seegmiller, and Jeanne Day. 1977. On the coding of spatial information. Memory & Cognition 5, 1, 10--16.

[22]

Miguel A. Nacenta, Yemliha Kamber, Yizhou Qiang, and Per Ola Kristensson. 2013. Memorability of Pre-designed and User-defined Gesture Sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 1099--1108.

Digital Library

[23]

Allan Paivio. 1965. Abstractness, imagery, and meaningfulness in paired-associate learning. Journal of Verbal Learning and Verbal Behavior 4, 1, 32 -- 38.

[24]

Simon T. Perrault, Eric Lecolinet, Yoann Pascal Bourse, Shengdong Zhao, and Yves Guiard. 2015. Physical loci: Leveraging spatial, object and semantic memory for command selection. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, 299--308.

Digital Library

[25]

George Robertson, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten van Dantzich. 1998. Data mountain: Using spatial memory for document management. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST '98). ACM, 153--162.

Digital Library

[26]

Anne Roudaut, Gilles Bailly, Eric Lecolinet, and Laurence Nigay. 2009. Leaf Menus: Linear Menus with Stroke Shortcuts for Small Handheld Devices. In Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I (INTERACT '09). Springer-Verlag, 616--619.

Digital Library

[27]

Vit Rusnak, Caroline Appert, Olivier Chapuis, and Emmanuel Pietriga. 2018. Designing coherent gesture sets for multi-scale navigation on tabletops. In Proceedings of the 36th international conference on Human factors in computing systems (CHI '18). ACM, 142:1--142:12.

Digital Library

[28]

Joey Scarr, Andy Cockburn, Carl Gutwin, and Andrea Bunt. 2012. Improving command selection with CommandMaps. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, 257--266.

Digital Library

[29]

Joey Scarr, Andy Cockburn, Carl Gutwin, and Sylvain Malacria. 2013. Testing the robustness and performance of spatially consistent interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 3139--3148.

Digital Library

[30]

Richard A. Schmidt, Timothy Donald Lee, and others. 2005. Motor control and learning: A behavioral emphasis. Vol. 4. Human kinetics, Champaign, IL, USA.

[31]

Endel Tulving and others. 1972. Episodic and semantic memory. Organization of memory 1, 381--403.

[32]

Md. Sami Uddin, Carl Gutwin, and Andy Cockburn. 2017. The Effects of Artificial Landmarks on Learning and Performance in Spatial-Memory Interfaces. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, 3843--3855.

Digital Library

[33]

Md. Sami Uddin, Carl Gutwin, and Benjamin Lafreniere. 2016. Handmark menus: Rapid command selection and large command sets on multi-touch displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 5836--5848.

Digital Library

[34]

Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics bulletin 1, 6, 80--83.

[35]

Frances Yates. 1992. The Art of Memory . Random House UK, London.

[36]

Shengdong Zhao, Maneesh Agrawala, and Ken Hinckley. 2006. Zone and polygon menus: Using relative position to increase the breadth of multi-stroke marking menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, 1077--1086.

Digital Library

[37]

Shengdong Zhao and Ravin Balakrishnan. 2004. Simple vs. Compound Mark Hierarchical Marking Menus. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, 33--42.

Digital Library

[38]

Jingjie Zheng, Xiaojun Bi, Kun Li, Yang Li, and Shumin Zhai. 2018. M3 gesture menu: Design and experimental analyses of marking menus for touchscreen mobile interaction. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, Article 249, 14 pages.

Digital Library

Cited By

Sluÿters ASellier QVanderdonckt JParthiban VMaes P(2022)Consistent, Continuous, and Customizable Mid-Air Gesture Interaction for Browsing Multimedia Objects on Large DisplaysInternational Journal of Human–Computer Interaction10.1080/10447318.2022.207846439:12(2492-2523)Online publication date: 27-Jul-2022
https://doi.org/10.1080/10447318.2022.2078464
Fruchard BLecolinet EChapuis O(2020)Side-Crossing MenusProceedings of the ACM on Human-Computer Interaction10.1145/34273174:ISS(1-19)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3427317
Goguey AMalacria SCockburn AGutwin C(2019)Reducing error aversion to support novice-to-expert transitions with FastTapProceedings of the 31st Conference on l'Interaction Homme-Machine10.1145/3366550.3372247(1-10)Online publication date: 10-Dec-2019
https://dl.acm.org/doi/10.1145/3366550.3372247
Show More Cited By

Index Terms

How Memorizing Positions or Directions Affects Gesture Learning?
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input

Recommendations

Command memorization: spatial positions versus directional gestures
IHM '18: Proceedings of the 30th Conference on l'Interaction Homme-Machine

To execute a command, a user can make a selection in a menu using pointing actions, or, for example, use shortcuts such as gestures. To go faster, she can learn the positions to point or gestures to perform. In this paper, we present a study comparing ...
Gaze+Gesture: Expressive, Precise and Targeted Free-Space Interactions
ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Humans rely on eye gaze and hand manipulations extensively in their everyday activities. Most often, users gaze at an object to perceive it and then use their hands to manipulate it. We propose applying a multimodal, gaze plus free-space gesture ...
Look and lean: accurate head-assisted eye pointing
ETRA '14: Proceedings of the Symposium on Eye Tracking Research and Applications

Compared to the mouse, eye pointing is inaccurate. As a consequence, small objects are difficult to point by gaze alone. We suggest using a combination of eye pointing and subtle head movements to achieve accurate hands-free pointing in a conventional ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISS '18: Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces

November 2018

499 pages

ISBN:9781450356947

DOI:10.1145/3279778

General Chairs:
Hideki Koike
Tokyo Institute of Technology, Japan
,
Carlo Ratti
Massachusetts Institute of Technology, USA
,
Yuichiro Takeuchi
Sony Computer Science Laboratories Inc., Japan
,
Program Chairs:
Kentaro Fukuchi
Meiji University, Japan
,
Stacey Scott
University of Guelph, Canada
,
Diego Martinez Plasencia
University of Sussex, United Kingdom

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Note

Funding Sources

Agence Nationale de la Recherche

Conference

ISS '18

Sponsor:

SIGCHI

ISS '18: 2018 ACM International Conference on Interactive Surfaces and Spaces

November 25 - 28, 2018

Tokyo, Japan

Acceptance Rates

ISS '18 Paper Acceptance Rate 28 of 105 submissions, 27%;

Overall Acceptance Rate 147 of 533 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
165
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sluÿters ASellier QVanderdonckt JParthiban VMaes P(2022)Consistent, Continuous, and Customizable Mid-Air Gesture Interaction for Browsing Multimedia Objects on Large DisplaysInternational Journal of Human–Computer Interaction10.1080/10447318.2022.207846439:12(2492-2523)Online publication date: 27-Jul-2022
https://doi.org/10.1080/10447318.2022.2078464
Fruchard BLecolinet EChapuis O(2020)Side-Crossing MenusProceedings of the ACM on Human-Computer Interaction10.1145/34273174:ISS(1-19)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3427317
Goguey AMalacria SCockburn AGutwin C(2019)Reducing error aversion to support novice-to-expert transitions with FastTapProceedings of the 31st Conference on l'Interaction Homme-Machine10.1145/3366550.3372247(1-10)Online publication date: 10-Dec-2019
https://dl.acm.org/doi/10.1145/3366550.3372247
Vanderdonckt JPetit É(2019)G-Menu: A Keyword-by-Gesture Based Dynamic Menu Interface for SmartphonesHuman-Computer Interaction. Recognition and Interaction Technologies10.1007/978-3-030-22643-5_8(99-114)Online publication date: 27-Jun-2019
https://doi.org/10.1007/978-3-030-22643-5_8
Fruchard BLecolinet EChapuis OKubicki SGirard PCalvary GLavoué E(2018)Command memorizationProceedings of the 30th Conference on l'Interaction Homme-Machine10.1145/3286689.3286700(92-99)Online publication date: 23-Oct-2018
https://dl.acm.org/doi/10.1145/3286689.3286700

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents