research-article

Open access

Optimizing Touchscreen Keyboards for Gesture Typing

Authors:

Brian A. Smith,

Shumin ZhaiAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 3365 - 3374

https://doi.org/10.1145/2702123.2702357

Published: 18 April 2015 Publication History

Abstract

Despite its growing popularity, gesture typing suffers from a major problem not present in touch typing: gesture ambiguity on the Qwerty keyboard. By applying rigorous mathematical optimization methods, this paper systematically investigates the optimization space related to the accuracy, speed, and Qwerty similarity of a gesture typing keyboard. Our investigation shows that optimizing the layout for gesture clarity (a metric measuring how unique word gestures are on a keyboard) drastically improves the accuracy of gesture typing. Moreover, if we also accommodate gesture speed, or both gesture speed and Qwerty similarity, we can still reduce error rates by 52% and 37% over Qwerty, respectively. In addition to investigating the optimization space, this work contributes a set of optimized layouts such as GK-D and GK-T that can immediately benefit mobile device users.

Supplementary Material

MP4 File (p3365.mp4)

Download
131.24 MB

References

[1]

Android Developers. https://source.android.com.

[2]

Bi, X., Azenkot, S., Partridge, K., and Zhai, S. Octopus: evaluating touchscreen keyboard correction and recognition algorithms via "remulation." In Proc. CHI 2013, ACM Press (2013), 543--552.

Digital Library

[3]

Bi, X., Chelba, C., Ouyang, T., Partridge, K., and Zhai, S. Bimanual gesture keyboard. In Proc. UIST 2012, ACM Press (2012), 137--146.

Digital Library

[4]

Bi, X., Ouyang, T., and Zhai, S. Both complete and correct? Multiobjective optimization of a touchscreen keyboard. In Proc. CHI 2014, ACM Press (2014), 2297--2306.

Digital Library

[5]

Bi, X., Smith, B.A., and Zhai, S. Multilingual touchscreen keyboard design and optimization. Human-Computer Interaction 27, 4 (2012), 352--382.

[6]

Bi, X., Smith, B.A., and Zhai, S. Quasi-Qwerty soft keyboard optimization. In Proc. CHI 2010, ACM Press (2010), 283--286.

Digital Library

[7]

Cao, X. and Zhai, S. Modeling human performance of pen stroke gestures. In Proc. CHI 2007, ACM Press (2007), 1495--1504.

Digital Library

[8]

Dunlop, M.D. and Levine, J. Multidimensional Pareto optimization of touchscreen keyboards for speed, familiarity, and improved spell checking. In Proc. CHI 2012, ACM Press (2012), 2669--2678.

Digital Library

[9]

Getschow, C.O., Rosen, M.J., and GoodenoughTrepagnier, C. A systematic approach to design a minimum distance alphabetical keyboard. In Proc. RESNA 1986, RESNA (1986), 396--398.

[10]

Hastings, W.K. Monte Carlo sampling methods using Markov chains and their applications. In Biometrika 57, 1 (1970), 97--109.

[11]

Kristensson, P.O. and Zhai, S. SHARK2 : a large vocabulary shorthand writing system for pen-based computers. In Proc. UIST 2004, ACM Press (2004), 43--52.

Digital Library

[12]

Lewis, J.R., Kennedy, P.J., and LaLomia, M.J. Development of a digram-based typing key layout for single finger/stylus input. In Proc. Hum. Fact. Ergon. Soc. 1999, Sage Publications (1999), 415--419.

[13]

MacKensie, I.S. and Zhang, S. The design and evaluation of a high-performance soft keyboard. In Proc. CHI 1999, ACM Press (1999), 25--31.

Digital Library

[14]

Mankoff, J. and Abowd, G.D. Cirrin: a word-level unistroke keyboard for pen input. In Proc. UIST 1998, ACM Press (1998), 213--214.

Digital Library

[15]

Oulasvirta, A., Reichel, A., Li, W., Zhang, Y., Bachnynskyi, M., Vertanen, K., and Kristensson, P.O. Improving two-thumb text entry on touchscreen devices. In Proc. CHI 2013, ACM Press (2013), 2765--2774.

Digital Library

[16]

Nexus 5 -- Google. http://www.google.com/nexus/5/

[17]

Perlin, K. Quikwriting: continuous stylus-based text entry. In Proc. UIST 1998, ACM Press (1998), 215--216.

Digital Library

[18]

Rick, J. Performance optimizations of virtual keyboards for stroke-based text entry on a touch-based tabletop. In Proc. UIST 2010, ACM Press (2010), 77--86.

Digital Library

[19]

Yamada, H. A historical study of typewriters and typing methods: from the position of planning Japanese parallels. Information Processing 2, 4 (1980), 175--202.

[20]

Zhai, S., Hunter, M., and Smith, B.A. The Metropolis keyboard-an exploration of quantitative techniques for virtual keyboard design. In Proc. UIST 2000, ACM Press (2000), 119--128.

Digital Library

[21]

Zhai, S., Hunter, M., and Smith, B.A. Performance optimization of virtual keyboards. Human-Computer Interaction 17, 2 (2002), 229--269.

[22]

Zhai, S. and Kristensson, P.O. Interlaced QWERTY -- accommodating ease of visual search and input flexibility in shape writing. In Proc. CHI 2008, ACM Press (2008), 593--596.

Digital Library

[23]

Zhai, S. and Kristensson, P.O. Introduction to shape writing. Text Entry Systems: Mobility, Accessibility, Universality. Morgan Kaufmann, 2010. 139--158.

[24]

Zhai, S. and Kristensson, P.O. Shorthand writing on stylus keyboard. In Proc. CHI 2003, ACM Press (2003), 97--104.

Digital Library

[25]

Zhai, S. and Kristensson, P.O. The word-gesture keyboard: reimagining keyboard interaction. Communications of the ACM 55, 9 (2012), 91--101.

Digital Library

Cited By

Yoshida SKoyama YUshiku Y(2024)Toward AI-Mediated Avatar-Based Telecommunication: Investigating Visual Impression of Switching Between User- and AI-Controlled Avatars in Video ChatIEEE Access10.1109/ACCESS.2024.344123312(113372-113383)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3441233
Johns CEvangelista Belo JFeit AKlokmose CPfeuffer K(2023)Towards Flexible and Robust User Interface Adaptations With Multiple ObjectivesProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606799(1-17)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606799
Johns CEvangelista Belo JKlokmose CPfeuffer K(2023)Pareto Optimal Layouts for Adaptive Mixed RealityExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585732(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585732
Show More Cited By

Index Terms

Optimizing Touchscreen Keyboards for Gesture Typing
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Vulture: a mid-air word-gesture keyboard
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Word-gesture keyboards enable fast text entry by letting users draw the shape of a word on the input surface. Such keyboards have been used extensively for touch devices, but not in mid-air, even though their fluent gestural input seems well suited for ...
Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

The popularity of mobile devices with large screens is making single-handed interaction difficult. We propose and evaluate a novel design point around a tilt-based text entry technique which supports single handed usage. Our technique is based on the ...
Why Is Gesture Typing Promising for Older Adults?: Comparing Gesture and Tap Typing Behavior of Older with Young Adults
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility

Gesture typing has been a widely adopted text entry method on touchscreen devices. We have conducted a study to understand whether older adults could gesture type, how they type, what are the strengths and weaknesses of gesture typing, and how to further ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Copyright © 2015 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
2,294
Total Downloads

Downloads (Last 12 months)202
Downloads (Last 6 weeks)28

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yoshida SKoyama YUshiku Y(2024)Toward AI-Mediated Avatar-Based Telecommunication: Investigating Visual Impression of Switching Between User- and AI-Controlled Avatars in Video ChatIEEE Access10.1109/ACCESS.2024.344123312(113372-113383)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3441233
Johns CEvangelista Belo JFeit AKlokmose CPfeuffer K(2023)Towards Flexible and Robust User Interface Adaptations With Multiple ObjectivesProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606799(1-17)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606799
Johns CEvangelista Belo JKlokmose CPfeuffer K(2023)Pareto Optimal Layouts for Adaptive Mixed RealityExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585732(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585732
Chu JAn DMa YCui WZhai SGu XBi X(2023)WordGesture-GAN: Modeling Word-Gesture Movement with Generative Adversarial NetworkProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581279(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581279
Kuang EChen RFan M(2023)Enhancing Older Adults’ Gesture Typing Experience Using the T9 Keyboard on Small Touchscreen DevicesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581105(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581105
Lee LBraud THui P(2023)Embodied Interaction on Constrained Interfaces for Augmented RealitySpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_10(239-271)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_10
He ZLutteroth CPerlin K(2022)TapGazer: Text Entry with Finger Tapping and Gaze-directed Word SelectionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501838(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501838
Dube TJohnson KArif A(2022)Shapeshifter: Gesture Typing in Virtual Reality with a Force-based Digital ThimbleExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519679(1-9)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519679
Shen JHu JDudley JKristensson P(2022)Personalization of a Mid-Air Gesture Keyboard using Multi-Objective Bayesian Optimization2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00088(702-710)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00088
Turner CChaparro BSogaard IHe J(2021)The Effects of Keyboard Layout and Size on Smartphone Typing PerformanceProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132064123764:1(985-989)Online publication date: 9-Feb-2021
https://doi.org/10.1177/1071181320641237
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten