research-article

BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb's Muscle Memory

Authors:

Shengdong ZhaoAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 1, Issue 2

Article No.: 18, Pages 1 - 24

https://doi.org/10.1145/3090083

Published: 30 June 2017 Publication History

Abstract

Eyes-free input is desirable for ubiquitous computing, since interacting with mobile and wearable devices often competes for visual attention with other devices and tasks. In this paper, we explore eyes-free typing on a touchpad using one thumb, wherein a user taps on an imaginary QWERTY keyboard while receiving text feedback on a separate screen. Our hypothesis is that users can transfer their typing ability obtained from visible keyboards to eyes-free use. We propose two statistical decoding algorithms to infer users’ eyes-free input: the absolute algorithm and the relative algorithm. The absolute algorithm infers user input based on the absolute position of touch endpoints, while the relative algorithm infers based on the vectors between successive touch endpoints. Evaluation results showed users could achieve satisfying performance with both algorithms. Text entry rate was 17-23 WPM (words per minute) depending on the algorithm used. In comparison, a baseline cursor-based text entry method yielded only 7.66 WPM. In conclusion, our research demonstrates for the first time the feasibility of thumb-based eyes-free typing, which provides a new possibility for text entry on ubiquitous computing platforms such as smart TVs and HMDs.

Supplementary Material

lu (lu.zip)

Supplemental movie, appendix, image and software files for, BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb's Muscle Memory

Download
56.65 MB

References

[1]

2011. The Open American National Corpus (OANC). Website. (7 November 2011). http://www.anc.org/.

[2]

Shiri Azenkot, Jacob O. Wobbrock, Sanjana Prasain, and Richard E. Ladner. 2012. Input Finger Detection for Nonvisual Touch Screen Text Entry in Perkinput. In Proceedings of Graphics Interface 2012 (GI ‘12). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 121--129. http://dl.acm.org/citation.cfm?id=2305276.2305297

Digital Library

[3]

Shiri Azenkot and Shumin Zhai. 2012. Touch Behavior with Different Postures on Soft Smartphone Keyboards. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘12). ACM, New York, NY, USA, 251--260.

Digital Library

[4]

Nikola Banovic, Koji Yatani, and Khai N Truong. 2013. Escape-keyboard: A sight-free one-handed text entry method for mobile touch-screen devices. International Journal of Mobile Human Computer Interaction (IJMHCI) 5, 3 (2013), 42--61.

Digital Library

[5]

Aurora Barrero, David Melendi, Xabiel G Pañeda, Roberto García, and Sergio Cabrero. 2014. An empirical investigation into text input methods for interactive digital television applications. International Journal of Human-Computer Interaction 30, 4 (2014), 321--341.

[6]

Matthew N Bonner, Jeremy T Brudvik, Gregory D Abowd, and W Keith Edwards. 2010. No-look notes: accessible eyes-free multi-touch text entry. In International Conference on Pervasive Computing. Springer, 409--426.

Digital Library

[7]

James Clawson, Kent Lyons, Thad Starner, and Edward Clarkson. 2005. The impacts of limited visual feedback on mobile text entry for the twiddler and mini-qwerty keyboards. In Ninth IEEE International Symposium on Wearable Computers (ISWC'05). IEEE, 170--177.

Digital Library

[8]

Leah Findlater, Jacob O. Wobbrock, and Daniel Wigdor. 2011. Typing on Flat Glass: Examining Ten-finger Expert Typing Patterns on Touch Surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11). ACM, New York, NY, USA, 2453--2462.

Digital Library

[9]

Jérémie Gilliot, Géry Casiez, and Nicolas Roussel. 2014. Impact of Form Factors and Input Conditions on Absolute Indirect-touch Pointing Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 723--732.

Digital Library

[10]

Mayank Goel, Leah Findlater, and Jacob Wobbrock. 2012. WalkType: Using Accelerometer Data to Accomodate Situational Impairments in Mobile Touch Screen Text Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘12). ACM, New York, NY, USA, 2687--2696.

Digital Library

[11]

Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language Modeling for Soft Keyboards. In Proceedings of the 7th International Conference on Intelligent User Interfaces (IUI ‘02). ACM, New York, NY, USA, 194--195.

Digital Library

[12]

Tovi Grossman, Xiang Anthony Chen, and George Fitzmaurice. 2015. Typing on Glasses: Adapting Text Entry to Smart Eyewear. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ‘15). ACM, New York, NY, USA, 144--152.

Digital Library

[13]

Amy K Karlson and Benjamin B Bederson. 2007. ThumbSpace: generalized one-handed input for touchscreen-based mobile devices. In IFIP Conference on Human-Computer Interaction. Springer, 324--338.

Digital Library

[14]

Per-Ola Kristensson and Shumin Zhai. 2004. SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST ‘04). ACM, New York, NY, USA, 43--52.

Digital Library

[15]

Per-Ola Kristensson and Shumin Zhai. 2005. Relaxing Stylus Typing Precision by Geometric Pattern Matching. In Proceedings of the 10th International Conference on Intelligent User Interfaces (IUI ‘05). ACM, New York, NY, USA, 151--158.

Digital Library

[16]

Kevin A. Li, Patrick Baudisch, and Ken Hinckley. 2008. Blindsight: Eyes-free Access to Mobile Phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘08). ACM, New York, NY, USA, 1389--1398.

Digital Library

[17]

Kent Lyons, Thad Starner, Daniel Plaisted, James Fusia, Amanda Lyons, Aaron Drew, and E. W. Looney. 2004. Twiddler Typing: One-handed Chording Text Entry for Mobile Phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 04). ACM, New York, NY, USA, 671--678.

Digital Library

[18]

I Scott MacKenzie, Mauricio H Lopez, and Steven Castelluci. 2009. Text entry with the Apple iPhone and the Nintendo Wii. Proceedings of CHI2009 (2009).

[19]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI ‘03 Extended Abstracts on Human Factors in Computing Systems (CHI EA ‘03). ACM, New York, NY, USA, 754--755.

Digital Library

[20]

Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: A Mid-air Word-gesture Keyboard. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 1073--1082.

Digital Library

[21]

Antti Oulasvirta, Sakari Tamminen, Virpi Roto, and Jaana Kuorelahti. 2005. Interaction in 4-second Bursts: The Fragmented Nature of Attentional Resources in Mobile HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘05). ACM, New York, NY, USA, 919--928.

Digital Library

[22]

Pekka Parhi, Amy K. Karlson, and Benjamin B. Bederson. 2006. Target Size Study for One-handed Thumb Use on Small Touchscreen Devices. In Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘06). ACM, New York, NY, USA, 203--210.

Digital Library

[23]

Daniel R. Rashid and Noah A. Smith. 2008. Relative Keyboard Input System. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI ‘08). ACM, New York, NY, USA, 397--400.

Digital Library

[24]

Oliver Schoenleben and Antti Oulasvirta. 2013. Sandwich Keyboard: Fast Ten-finger Typing on a Mobile Device with Adaptive Touch Sensing on the Back Side. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘13). ACM, New York, NY, USA, 175--178.

Digital Library

[25]

Caleb Southern, James Clawson, Brian Frey, Gregory Abowd, and Mario Romero. 2012. An Evaluation of BrailleTouch: Mobile Touchscreen Text Entry for the Visually Impaired. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘12). ACM, New York, NY, USA, 317--326.

Digital Library

[26]

Hussain Tinwala and I. Scott MacKenzie. 2010. Eyes-free Text Entry with Error Correction on Touchscreen Mobile Devices. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI ‘10). ACM, New York, NY, USA, 511--520.

Digital Library

[27]

Keith Vertanen, Haythem Memmi, and Per Ola Kristensson. 2013. The Feasibility of Eyes-free Touchscreen Keyboard Typing. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘13). ACM, New York, NY, USA, Article 69, 2 pages.

Digital Library

[28]

Daniel Vogel and Patrick Baudisch. 2007. Shift: A Technique for Operating Pen-based Interfaces Using Touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘07). ACM, New York, NY, USA, 657--666.

Digital Library

[29]

Yuntao Wang, Chun Yu, Jie Liu, and Yuanchun Shi. 2013. Understanding Performance of Eyes-free, Absolute Position Control on Touchable Mobile Phones. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘13). ACM, New York, NY, USA, 79--88.

Digital Library

[30]

Jacob O. Wobbrock and Brad A. Myers. 2006. Analyzing the Input Stream for Character- Level Errors in Unconstrained Text Entry Evaluations. ACM Trans. Comput.-Hum. Interact. 13, 4 (Dec. 2006), 458--489.

Digital Library

[31]

Bo Yi, Xiang Cao, Morten Fjeld, and Shengdong Zhao. 2012. Exploring User Motivations for Eyes-free Interaction on Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘12). ACM, New York, NY, USA, 2789--2792.

Digital Library

[32]

Chun Yu, Ke Sun, Mingyuan Zhong, Xincheng Li, Peijun Zhao, and Yuanchun Shi. 2016. One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 71--82.

Digital Library

[33]

Shengdong Zhao, Pierre Dragicevic, Mark Chignell, Ravin Balakrishnan, and Patrick Baudisch. 2007. Earpod: Eyes-free Menu Selection Using Touch Input and Reactive Audio Feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘07). ACM, New York, NY, USA, 1395--1404.

Digital Library

Cited By

Wu JWang ZWang LDuan YLi J(2024)FanPad: A Fan Layout Touchpad Keyboard for Text Entry in VR2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00045(222-232)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00045
Shi YYi XLiang CQin YWang YYan YCheng ZZhu PZhang SLi YLiu YWang XChen JZhou WWang YZhao DDu F(2024)HCI Research and Innovation in China: A 10-Year PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2323858(1-33)Online publication date: 22-Mar-2024
https://doi.org/10.1080/10447318.2024.2323858
Yi XLiang CChen HSong JYu CLi HShi Y(2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580829
Show More Cited By

Index Terms

BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb's Muscle Memory
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Text input
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing theory, concepts and paradigms
      1. Ubiquitous computing
    2. Ubiquitous and mobile devices
      1. Mobile devices

Recommendations

B2B-Swipe: Swipe Gesture for Rectangular Smartwatches from a Bezel to a Bezel
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We present B2B-Swipe, a single-finger swipe gesture for a rectangular smartwatch that starts at a bezel and ends at a bezel to enrich input vocabulary. There are 16 possible B2B-Swipes because a rectangular smartwatch has four bezels. Moreover, B2B-...
Vulture: a mid-air word-gesture keyboard
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Word-gesture keyboards enable fast text entry by letting users draw the shape of a word on the input surface. Such keyboards have been used extensively for touch devices, but not in mid-air, even though their fluent gestural input seems well suited for ...
Text Entry for Ultra-Small Touchscreens Using a Fixed Cursor and Movable Keyboard
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Emerging ultra-small wearables like smartwatches pose a design challenge for touch-based text entry, due to the "fat finger" problem, wherein users struggle to select elements much smaller than their fingers. In order to address the challenge, we ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 1, Issue 2

June 2017

665 pages

EISSN:2474-9567

DOI:10.1145/3120957

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2017

Accepted: 01 April 2017

Revised: 01 April 2017

Received: 01 February 2017

Published in IMWUT Volume 1, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Natural Science Foundation of China
National Key Research and Development Plan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
1,198
Total Downloads

Downloads (Last 12 months)113
Downloads (Last 6 weeks)11

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu JWang ZWang LDuan YLi J(2024)FanPad: A Fan Layout Touchpad Keyboard for Text Entry in VR2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00045(222-232)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00045
Shi YYi XLiang CQin YWang YYan YCheng ZZhu PZhang SLi YLiu YWang XChen JZhou WWang YZhao DDu F(2024)HCI Research and Innovation in China: A 10-Year PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2323858(1-33)Online publication date: 22-Mar-2024
https://doi.org/10.1080/10447318.2024.2323858
Yi XLiang CChen HSong JYu CLi HShi Y(2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580829
Liu ZHe JFeng JZhou J(2023)PrinTypeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35694916:4(1-31)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569491
Li ZYu CGu YShi Y(2023)ResType: Invisible and Adaptive Tablet Keyboard Leveraging Resting FingersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581055(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581055
Faleel SLiu YCody RRey BDu LYu JHuang DIrani PLi W(2023)T-Force: Exploring the Use of Typing Force for Three State Virtual KeyboardsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580915(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580915
Iwasaki MHara SAbe M(2023)Speech Synthesis Using Ambiguous Inputs From Wearable Keyboards2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)10.1109/APSIPAASC58517.2023.10317228(1172-1178)Online publication date: 31-Oct-2023
https://doi.org/10.1109/APSIPAASC58517.2023.10317228
Sun AWang LLeng JKei Im S(2023)Light-Occlusion Text Entry in Mixed RealityInternational Journal of Human–Computer Interaction10.1080/10447318.2023.2285646(1-16)Online publication date: 28-Nov-2023
https://doi.org/10.1080/10447318.2023.2285646
Pooripanyakun MWodehouse AMehnen J(2023)The role of interface configuration on performance accuracy in eyes-free touchscreen interactionUniversal Access in the Information Society10.1007/s10209-023-01057-zOnline publication date: 31-Oct-2023
https://doi.org/10.1007/s10209-023-01057-z
Lee LBraud THui P(2023)Embodied Interaction on Constrained Interfaces for Augmented RealitySpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_10(239-271)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_10
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents