research-article

MarkPad: Augmenting Touchpads for Command Selection

Authors:

Bruno Fruchard,

Eric Lecolinet,

Olivier ChapuisAuthors Info & Claims

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Pages 5630 - 5642

https://doi.org/10.1145/3025453.3025486

Published: 02 May 2017 Publication History

Abstract

We present MarkPad, a novel interaction technique taking advantage of the touchpad. MarkPad allows creating a large number of size-dependent gestural shortcuts that can be spatially organized as desired by the user. It relies on the idea of using visual or tactile marks on the touchpad or a combination of them. Gestures start from a mark on the border and end on another mark anywhere. MarkPad does not conflict with standard interactions and provides a novice mode that acts as a rehearsal of the expert mode. A first study showed that an accuracy of 95% could be achieved for a dense configuration of tactile and/or visual marks allowing many gestures. Performance was 5% lower in a second study where the marks were only on the borders. A last study showed that borders are rarely used, even when the users are unaware of the technique. Finally, we present a working prototype and briefly report on how it was used by two users for a few months.

Supplementary Material

ZIP File (pn1170-file4.zip)

Download
1.60 KB

suppl.mov (pn1170.mp4)

Supplemental video

Download
28.20 MB

suppl.mov (pn1170p.mp4)

Supplemental video

Download
2.10 MB

References

[1]

Actronika. Tactronik. Retrieved January 2017 from http://www.actronika.com/

[2]

Georg Apitz and François Guimbretière. 2004. Crossy: A crossing-based drawing application. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, 3--12.

Digital Library

[3]

Caroline Appert and Shumin Zhai. 2009. Using strokes as command shortcuts: Cognitive benefits and toolkit support. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, 2289--2298.

Digital Library

[4]

A.D. Baddeley. 2013. Essentials of Human Memory (Classic Edition). Psychology Press, East Sussex, UK.

[5]

Gilles Bailly, Eric Lecolinet, and Laurence Nigay. 2008. Flower menus: A new type of marking menu with large menu breadth, within groups and efficient expert mode memorization. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '08). ACM, 15--22.

Digital Library

[6]

Gilles Bailly, Thomas Pietrzak, Jonathan Deber, and Daniel J. Wigdor. 2013. Métamorphe: Augmenting hotkey usage with actuated keys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 563--572.

Digital Library

[7]

Olivier Bau and Wendy E. Mackay. 2008. Octopocus: A dynamic guide for learning gesture-based command sets. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08). ACM, 37--46.

Digital Library

[8]

Olivier Bau, Ivan Poupyrev, Ali Israr, and Chris Harrison. 2010. Teslatouch: Electrovibration for touch surfaces. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, 283--292.

Digital Library

[9]

Mathieu Berthellemy, Elodie Cayez, Marwan Ajem, Gilles Bailly, Sylvain Malacria, and Eric Lecolinet. 2015. SpotPad, LociPad, ChordPad and InOutPad: Investigating gesture-based input on touchpad. In Proceedings of the 27th Conference on L'Interaction Homme-Machine (IHM '15). ACM, Article 4, 8 pages.

Digital Library

[10]

Gordon H. Bower, Michal C. Clark, Alan M. Lesgold, and David Winzenz. 1969. Hierarchical retrieval schemes in recall of categorized word lists. Journal of Verbal Learning and Verbal Behavior 8, 3, 323--343.

[11]

Andrew Bragdon, Eugene Nelson, Yang Li, and Ken Hinckley. 2011. Experimental analysis of touch-screen gesture designs in mobile environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, 403--412.

Digital Library

[12]

Géry Casiez, Daniel Vogel, Qing Pan, and Christophe Chaillou. 2007. RubberEdge: Reducing clutching by combining position and rate control with elastic feedback. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST '07). ACM, 129--138.

Digital Library

[13]

Olivier Chapuis and Nicolas Roussel. 2007. Copy-and-paste between overlapping windows. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, 201--210.

Digital Library

[14]

Christian Corsten, Christian Cherek, Thorsten Karrer, and Jan Borchers. 2015. Hapticase: Back-of-device tactile landmarks for eyes-free absolute indirect touch. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, 2171--2180.

Digital Library

[15]

Mary Czerwinski, Maarten Van Dantzich, George Robertson, and Hunter Hoffman. 1999. The contribution of thumbnail image, mouse-over text and spatial location memory to web page retrieval in 3D. In Proceedings of INTERACT, Vol. 99. ISO Press, 163--170. http://www.msr-waypoint.net/en-us/um/people/ marycz/interact99.pdf

[16]

Jérémie Gilliot, Géry Casiez, and Nicolas Roussel. 2014. Impact of form factors and input conditions on absolute indirect-touch pointing tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, 723--732.

Digital Library

[17]

Tovi Grossman, Xiang Anthony Chen, and George Fitzmaurice. 2015. Typing on glasses: Adapting text entry to smart eyewear. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). ACM, 144--152.

Digital Library

[18]

Carl Gutwin, Andy Cockburn, Joey Scarr, Sylvain Malacria, and Scott C. Olson. 2014. Faster command selection on tablets with FastTap. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, 2617--2626.

Digital Library

[19]

Chris Harrison and Scott E. Hudson. 2009. Providing dynamically changeable physical buttons on a visual display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, 299--308.

Digital Library

[20]

Andreas Hegenberg. BetterTouchTool application. Retrieved January 2017 from https://www.boastr.net

[21]

Mohit Jain and Ravin Balakrishnan. 2012. User learning and performance with bezel menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, 2221--2230.

Digital Library

[22]

Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2012. Tangible remote controllers for wall-size displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, 2865--2874.

Digital Library

[23]

Yvonne Jansen, Thorsten Karrer, and Jan Borchers. 2010. Mudpad: A tactile memory game. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10). ACM, 306--306.

Digital Library

[24]

William P. Jones and Susan T. Dumais. 1986. The spatial metaphor for user interfaces: Experimental tests of reference by location versus name. ACM Trans. Inf. Syst. 4, 1, 42--63.

Digital Library

[25]

Yuki Kubo, Buntarou Shizuki, and Jiro Tanaka. 2016. B2B-Swipe: Swipe gesture for rectangular smartwatches from a bezel to a bezel. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 3852--3856.

Digital Library

[26]

Gordon Kurtenbach and William Buxton. 1991. Issues in combining marking and direct manipulation techniques. In Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology (UIST '91). ACM, 137--144.

Digital Library

[27]

Gordon Kurtenbach and William Buxton. 1993. The limits of expert performance using hierarchic marking menus. In Proceedings of the INTERCHI '93 Conference on Human Factors in Computing Systems (INTERCHI '93). IOS Press, 482--487. http://dl.acm.org/citation.cfm?id=164632.164977

Digital Library

[28]

Benjamin Lafreniere, Carl Gutwin, Andy Cockburn, and Tovi Grossman. 2016. Faster command selection on touchscreen watches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 4663--4674.

Digital Library

[29]

MagicPrefs. Magic Trackpad application. Retrieved January 2017 from http://magicprefs.com

[30]

George A Miller. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review 63, 2, 81.

[31]

Mathieu Nancel and Michel Beaudouin-Lafon. 2008. Extending Marking Menus with Integral Dimensions: Application to the Dartboard Menu. Technical Report Technical report 1503. LRI.

[32]

K. L. Norman. 1991. The Psychology of Menu Selection: Designing Cognitive Control at the Human/Computer Interface. Ablex Publishing Corp., Norwood, NJ, USA.

Digital Library

[33]

Pekka Parhi, Amy K. Karlson, and Benjamin B. Bederson. 2006. Target size study for one-handed thumb use on small touchscreen devices. In Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '06). ACM, 203--210.

Digital Library

[34]

Thomas V. Perneger. 1998. What's wrong with bonferroni adjustments. British Medical Journal 316, 7139, 1236--1238.

[35]

Simon T. Perrault, Eric Lecolinet, Yoann Pascal Bourse, Shengdong Zhao, and Yves Guiard. 2015. Physical loci: Leveraging spatial, object and semantic memory for command selection. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, 299--308.

Digital Library

[36]

Jeffrey Pierce, Matthew Conway, Maarten van Dantzich, and George Robertson. 1999. Toolspaces and glances: storing, accessing, and retrieving objects in 3d desktop applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, 163--168.

Digital Library

[37]

Volker Roth and Thea Turner. 2009. Bezel swipe: Conflict-free scrolling and multiple selection on mobile touch screen devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, 1523--1526.

Digital Library

[38]

Quentin Roy, Sylvain Malacria, Yves Guiard, Eric Lecolinet, and James Eagan. 2013. Augmented letters: Mnemonic gesture-based shortcuts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 2325--2328.

Digital Library

[39]

Joey Scarr, Andy Cockburn, and Carl. Gutwin. 2012a. Supporting and exploiting spatial memory in user interfaces. Foundations and Trends in Human-Computer Interaction 6, 1, 1--84.

Digital Library

[40]

Joey Scarr, Andy Cockburn, Carl Gutwin, and Andrea Bunt. 2012b. Improving command selection with CommandMaps. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, 257--266.

Digital Library

[41]

Joey Scarr, Andy Cockburn, Carl Gutwin, and Sylvain Malacria. 2013. Testing the robustness and performance of spatially consistent interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 3139--3148.

Digital Library

[42]

Katherine Schramm, Carl Gutwin, and Andy Cockburn. 2016. Supporting transitions to expertise in hidden toolbars. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 4687--4698.

Digital Library

[43]

Marcos Serrano, Eric Lecolinet, and Yves Guiard. 2013. Bezel-tap gestures: Quick activation of commands from sleep mode on tablets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 3027--3036.

Digital Library

[44]

Tobii Eyetracking. Tobii EyeX. Retrieved January 2017 from http://www.tobii.com/xperience/

[45]

Nicolas Villar, Shahram Izadi, Dan Rosenfeld, Hrvoje Benko, John Helmes, Jonathan Westhues, Steve Hodges, Eyal Ofek, Alex Butler, Xiang Cao, and Billy Chen. 2009. Mouse 2.0: Multi-touch meets the mouse. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, 33--42.

Digital Library

[46]

Daniel Vogel and Patrick Baudisch. 2007. Shift: A technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, 657--666.

Digital Library

[47]

Julie Wagner, Eric Lecolinet, and Ted Selker. 2014. Multi-finger chords for hand-held tablets: Recognizable and memorable. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, 2883--2892.

Digital Library

[48]

Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the guessability of symbolic input. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, 1869--1872.

Digital Library

[49]

Shengdong Zhao, Maneesh Agrawala, and Ken Hinckley. 2006. Zone and polygon menus: Using relative position to increase the breadth of multi-stroke marking menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, 1077--1086.

Digital Library

Cited By

Lee JBaeck MYeo HStarner TWoo W(2024)GestureMark: Shortcut Input Technique using Smartwatch Touch Gestures for XR GlassesProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652941(63-71)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3652941
Gori JFruchard BBailly G(2024)Model-based Evaluation of Recall-based Interaction TechniquesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642637(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642637
Perelman GDubois ESerrano M(2024)Dual-Thumb pointing and command selection techniques for tabletsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103203184:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103203
Show More Cited By

Index Terms

MarkPad: Augmenting Touchpads for Command Selection
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Side-Crossing Menus: Enabling Large Sets of Gestures for Small Surfaces
ISS

Supporting many gestures on small surfaces allows users to interact remotely with complex environments such as smart homes, large remote displays, or virtual reality environments, and switching between them (e.g., AR setup in a smart home). Providing ...
Bezel-Tap gestures: quick activation of commands from sleep mode on tablets
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We present Bezel-Tap Gestures, a novel family of interaction techniques for immediate interaction on handheld tablets regardless of whether the device is alive or in sleep mode. The technique rests on the close succession of two input events: first a ...
HaptiCase: Back-of-Device Tactile Landmarks for Eyes-Free Absolute Indirect Touch
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Using a smartphone for touch input to control apps and games mirrored to a distant screen is difficult, as the user cannot see where she is touching while looking at the distant display. We present HaptiCase, an interaction technique that provides back-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

May 2017

7138 pages

ISBN:9781450346559

DOI:10.1145/3025453

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

DigiCosme

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
654
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee JBaeck MYeo HStarner TWoo W(2024)GestureMark: Shortcut Input Technique using Smartwatch Touch Gestures for XR GlassesProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652941(63-71)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3652941
Gori JFruchard BBailly G(2024)Model-based Evaluation of Recall-based Interaction TechniquesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642637(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642637
Perelman GDubois ESerrano M(2024)Dual-Thumb pointing and command selection techniques for tabletsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103203184:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103203
Perelman GDubois EProbst ASerrano M(2022)Visual Transitions around Tabletops in Mixed Reality: Study on a Visual Acquisition Task between Vertical Virtual Displays and Horizontal TabletopsProceedings of the ACM on Human-Computer Interaction10.1145/35677386:ISS(660-679)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567738
Probst APerelman GDubois ESerrano M(2022)Acquisition visuelle en réalité mixte, depuis et vers une table interactive : Analyse de la transition entre un écran virtuel et un écran physiqueProceedings of the 33rd Conference on l'Interaction Humain-Machine10.1145/3500866.3516377(1-17)Online publication date: 5-Apr-2022
https://dl.acm.org/doi/10.1145/3500866.3516377
McGill MBrewster SDe Sa Medeiros DBovet SGutierrez MKehoe A(2022)Creating and Augmenting Keyboards for Extended Reality with the Keyboard Augmentation ToolkitACM Transactions on Computer-Human Interaction10.1145/349049529:2(1-39)Online publication date: 16-Jan-2022
https://dl.acm.org/doi/10.1145/3490495
Safin SMaitrallin MFruchard BLecolinet E(2021)Processus d’appropriation et de mémorisation de raccourcis gestuels sur trackpad : Etude longitudinale des stratégies et usages des utilisateurs et impact d’une aide visuo-sémantiqueProceedings of the 32nd Conference on l'Interaction Homme-Machine10.1145/3450522.3451332(1-12)Online publication date: 13-Apr-2021
https://dl.acm.org/doi/10.1145/3450522.3451332
Keddisseh ESerrano MDubois E(2021)SmartCom: Exploiter un Smartphone pour Interagir avec les Barres d'OutilsProceedings of the 32nd Conference on l'Interaction Homme-Machine10.1145/3450522.3451244(1-10)Online publication date: 13-Apr-2021
https://dl.acm.org/doi/10.1145/3450522.3451244
Keddisseh ESerrano MDubois EKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)KeyTch: Combining the Keyboard with a Touchscreen for Rapid Command Selection on ToolbarsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445288(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445288
Cui WZhu SLi ZXu ZYang XRamakrishnan IBi XKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)BackSwipe: Back-of-device Word-Gesture Interaction on SmartphonesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445081(1-12)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445081
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents