research-article

The design and evaluation of multi-finger mouse emulation techniques

Authors:

Justin Matejka,

George FitzmauriceAuthors Info & Claims

CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1073 - 1082

https://doi.org/10.1145/1518701.1518865

Published: 04 April 2009 Publication History

Abstract

We explore the use of multi-finger input to emulate full mouse functionality, such as the tracking state, three buttons, and chording. We first present the design space for such techniques, which serves as a guide for the systematic investigation of possible solutions. We then perform a series of pilot studies to come up with recommendations for the various aspects of the design space. These pilot studies allow us to arrive at a recommended technique, the SDMouse. In a formal study, the SDMouse was shown to significantly improve performance in comparison to previously developed mouse emulation techniques.

Supplementary Material

JPG File (31.jpg)

Download
7.04 KB

JPG File (p1073.jpg)

Download
7.26 KB

FLV File (31.flv)

Download
18.31 MB

WMV File (p1073.wmv)

Download
11.98 MB

References

[1]

Albinsson, P. and Zhai, S. (2003). High precision touch screen interaction. ACM CHI. 105--112.

Digital Library

[2]

Balakrishnan, R., Baudel, T., Kurtenbach, G., and Fitzmaurice, G. (1997). The Rockin'Mouse: integral 3D manipulation on a plane. ACM CHI. 311--318.

Digital Library

[3]

Benko, H., Wilson, A., Baudisch, P. (2006). Precise Selection Techniques for Multi-Touch Screens. ACM CHI. 1263--1272.

Digital Library

[4]

Brandl, P., Forlines, C., Wigdor, D., Haller, M., and Shen, C. (2008). Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. AVI. 154--161.

Digital Library

[5]

Buxton, W., Hill, R., and Rowley, P. (1985). Issues and techniques in touch-sensitive tablet input. ACM SIGGRAPH, 215--224.

Digital Library

[6]

Dietz, P. and Leigh, D. (2001). DiamondTouch: a multi-user touch technology. ACM UIST. 219--226.

Digital Library

[7]

Echtler, F., Huber, M., and Klinker, G. (2008). Shadow tracking on multi-touch tables. AVI. 388--391.

Digital Library

[8]

Esenther, A. and Ryall, K. (2006). Fluid DTMouse: better mouse support for touch-based interactions. AVI. 112--115.

Digital Library

[9]

Fingerworks, Inc. (2008). User's Guide. http://www.fingerworks.com/gesture_guide_mouse.html

[10]

Fitzmaurice, G., Khan, A., Pieké, R., Buxton, B., and Kurtenbach, G. (2003). Tracking menus. ACM UIST. 71--79.

Digital Library

[11]

Forlines, C., Vogel, D., and Balakrishnan, R. (2006). HybridPointing: fluid switching between absolute and relative pointing with a direct input device. ACM UIST. 211--220.

Digital Library

[12]

Forlines, C., Wigdor, D., Shen, C., and Balakrishnan, R. (2007). Direct-touch vs. mouse input for tabletop displays. ACM CHI. 647--656.

Digital Library

[13]

Grossman, T., Hinckley, K., Baudisch, P., Agrawala, M., and Balakrishnan, R. (2006). Hover widgets: using the tracking state to extend the capabilities of pen-operated devices. ACM CHI. 861--870.

Digital Library

[14]

Grossman, T., Wigdor, D., and Balakrishnan, R. (2004). Multi-Finger Gestural Interaction with 3D Volumetric Displays. ACM UIST. 61--70.

Digital Library

[15]

Hager-Ross, C., Schieber, M. (2000). Quantifying the Independence of Human Finger Movements: Comparisions of Digits, Hands, and Movement Frequencies. J. of Neuroscience, 20(22), 8542--8550.

[16]

Han, J. Y. (2005). Low-cost multi-touch sensing through frustrated total internal reflection. ACM UIST 115--118.

Digital Library

[17]

Letessier, J. and Bérard, F. (2004). Visual tracking of bare fingers for interactive surfaces. ACM UIST. 119--122.

Digital Library

[18]

Malik, S. (2007). An Exploration of Multi-Finger Interaction on Multi-Touch Surfaces. Univ. of Toronto.

[19]

Malik, S., Ranjan, A., and Balakrishnan, R. (2006). Interacting with large displays from a distance with vision-tracked multi-finger gestural input. ACM UIST. 43--52.

Digital Library

[20]

Moscovich, T. (2007). Principles and Applications of Multi-touch Interaction. Brown University.

[21]

Moscovich, T. and Hughes, J. F. (2006). Multi-finger cursor techniques. Graphics Interface. 1--7.

Digital Library

[22]

Moscovich, T. and Hughes, J. F. (2008). Indirect mappings of multi-touch input using one and two hands. ACM CHI. 1275--1284.

Digital Library

[23]

Olwal, A., Feiner, S., and Heyman, S. (2008). Rubbing and tapping for precise and rapid selection on touch-screen displays. ACM CHI. 295--304.

Digital Library

[24]

Potter, R. L., Weldon, L. J., and Shneiderman, B. (1988). Improving the accuracy of touch screens: an experimental evaluation of three strategies. ACM CHI. 27--32.

Digital Library

[25]

Rekimoto, J. (2002). SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. ACM CHI. 113--120.

Digital Library

[26]

Sears, A. and Shneiderman, B. (1991) High Precision Touch-Screens: Design Strategies and Comparison with a Mouse, Int. Journ. of Man-Machine Studies, 43(4), 593--613.

Digital Library

[27]

Vogel, D. and Baudisch, P. (2007). Shift: a technique for operating pen-based interfaces using touch. ACM CHI. 657--666.

Digital Library

[28]

von Hardenberg, C. and Bérard, F. (2001). Bare-hand human-computer interaction. ACM PUI. 1--8.

Digital Library

[29]

Wu, M. and Balakrishnan, R. (2003). Multi-Finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. ACM UIST. 193--202.

Digital Library

[30]

Wu, M., Shen. C., Ryall, K., Forlines, C., Balakrishnan, R. (2006). Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces. IEEE TableTop. 183--190.

Digital Library

[31]

Zhang, Z., Wu, Y., Shan, Y., and Shafer, S. (2001). Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper. ACM PUI. 1--8.

Digital Library

Cited By

Lee HKo ITan YZhang DLi C(2019)Usability Impact of Occlusion-free Techniques on Commonly-used Multitouch ActionsProceedings of the 5th International ACM In-Cooperation HCI and UX Conference10.1145/3328243.3328256(96-105)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1145/3328243.3328256
Han TLiu JHasan KFan MKim JLi JFan XTian FLank EIrani PBrewster SFitzpatrick GCox AKostakos V(2019)PinchListProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300731(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300731
Tan YLee HKo IZhang DLi C(2019)A Multitouch Drawing Application with Occlusion-Free Interaction StrategiesHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29390-1_32(535-539)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29390-1_32
Show More Cited By

Index Terms

The design and evaluation of multi-finger mouse emulation techniques
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

Fluid DTMouse: better mouse support for touch-based interactions
AVI '06: Proceedings of the working conference on Advanced visual interfaces

Although computer mice have evolved physically (i.e., new form factors, multiple buttons, scroll-wheels), their basic metaphor remains the same: a single-point of interaction, with modifiers used to control the interaction. Many of today's novel input ...
3D positioning techniques for multi-touch displays
VRST '09: Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology

Multi-touch displays represent a promising technology for the display and manipulation of 3D data. To fully exploit their capabilities, appropriate interaction techniques must be designed. In this paper, we explore the design of free 3D positioning ...
The design and evaluation of multitouch marking menus
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Despite the considerable quantity of research directed towards multitouch technologies, a set of standardized UI components have not been developed. Menu systems provide a particular challenge, as traditional GUI menus require a level of pointing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2009

2426 pages

ISBN:9781605582467

DOI:10.1145/1518701

General Chairs:
Dan R. Olsen
Brigham Young University
,
Richard B. Arthur
Brigham Young University
,
Program Chairs:
Ken Hinckley
Microsoft Research
,
Meredith Ringel Morris
Microsoft Research
,
Scott Hudson
Carnegie Mellon University
,
Saul Greenberg
University of Calgary

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '09

Sponsor:

CHI '09: CHI Conference on Human Factors in Computing Systems

April 4 - 9, 2009

MA, Boston, USA

Acceptance Rates

CHI '09 Paper Acceptance Rate 277 of 1,130 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
1,990
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lee HKo ITan YZhang DLi C(2019)Usability Impact of Occlusion-free Techniques on Commonly-used Multitouch ActionsProceedings of the 5th International ACM In-Cooperation HCI and UX Conference10.1145/3328243.3328256(96-105)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1145/3328243.3328256
Han TLiu JHasan KFan MKim JLi JFan XTian FLank EIrani PBrewster SFitzpatrick GCox AKostakos V(2019)PinchListProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300731(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300731
Tan YLee HKo IZhang DLi C(2019)A Multitouch Drawing Application with Occlusion-Free Interaction StrategiesHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29390-1_32(535-539)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29390-1_32
Zhang YChen HLi WWang ZXin X(2018)Research on the Optimal Control-Display Ratio of Smart Phone Touch ScreenAdvances in Ergonomics in Design10.1007/978-3-319-94706-8_42(382-386)Online publication date: 24-Jun-2018
https://doi.org/10.1007/978-3-319-94706-8_42
ISHIHARA MISHIHARA Y(2017)A Shadow Cursor for Calibrating Screen Coordinates of Tabletop Displays and Its EvaluationIEICE Transactions on Information and Systems10.1587/transinf.2016EDP7331E100.D:6(1271-1279)Online publication date: 2017
https://doi.org/10.1587/transinf.2016EDP7331
Chen LWang LHuang WChen G(2017)A Comparative Study of Map Exploration Interfaces for Multi-Touch TabletopsInternational Journal of Human–Computer Interaction10.1080/10447318.2017.127889333:8(677-691)Online publication date: 11-Jan-2017
https://doi.org/10.1080/10447318.2017.1278893
Hayes SAdams J(2017)Control-display ratio enhancements for mobile interactionInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2016.10.00699(57-71)Online publication date: Mar-2017
https://doi.org/10.1016/j.ijhcs.2016.10.006
Hayes SAdams J(2016)Adaptive Control-Display Ratios for SmartphonesProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/154193121360118660:1(815-819)Online publication date: 8-Sep-2016
https://doi.org/10.1177/1541931213601186
Hayes SSteiger JAdams J(2016)Modelling touch-interaction time on smartphonesBehaviour & Information Technology10.1080/0144929X.2016.122146035:12(1022-1043)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1080/0144929X.2016.1221460
Leftheriotis IGiannakos MChorianopoulos KJaccheri LNunes NCostanza EOlivier PSchöning J(2015)Investigating the Potential of a Two-finger Chord Button in Multi-touch ApplicationsProceedings of the 2015 International Conference on Interactive Tabletops & Surfaces10.1145/2817721.2823503(337-342)Online publication date: 15-Nov-2015
https://dl.acm.org/doi/10.1145/2817721.2823503
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents