research-article

Public Access

Modeling Gliding-based Target Selection for Blind Touchscreen Users

Authors:

Ali Selman Aydin,

IV Ramakrishnan,

Antti Oulasvirta,

Xiaojun BiAuthors Info & Claims

MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction

Article No.: 29, Pages 1 - 14

https://doi.org/10.1145/3447526.3472022

Published: 27 September 2021 Publication History

Abstract

Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements with iterative corrections relying on non-visual feedback, and 2) multiple sub-movements separated by stops, and concatenated until the target is reached. Based on this finding, we propose the mixture pointing model, a model that relates movement time to distance and width of the target. The model outperforms extant models, improving R2 from 0.65 for Fitts’ law to 0.76, and is superior in cross-validation and information criteria. The model advances understanding of gliding-based target selection and serves as a tool for designing interface layouts for screen-reader based touch exploration.

References

[1]

H. Akaike. 1998. Information theory and an extension of the maximum likelihood principle. In Selected papers of hirotugu akaike. Springer, 199–213.

[2]

Apple. 2020. Vision Accessibility in iPhone. https://www.apple.com/accessibility/iphone/vision/

[3]

S. Azenkot, J. O. Wobbrock, 2012. Input finger detection for nonvisual touch screen text entry in Perkinput. In Proceedings of graphics interface 2012. 121–129.

[4]

P. Bach-y Rita, C. C. Collins, Vision substitution by tactile image projection. Nature 221, 5184 (1969), 963–964.

[5]

G. Bailly, A. Oulasvirta, 2014. Model of Visual Search and Selection Time in Linear Menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). Association for Computing Machinery, New York, NY, USA, 3865–3874. https://doi.org/10.1145/2556288.2557093

Digital Library

[6]

X. Bi, Y. Li, 2013. FFitts Law: Modeling Finger Touch with Fitts’ Law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’13). Association for Computing Machinery, New York, NY, USA, 1363–1372. https://doi.org/10.1145/2470654.2466180

Digital Library

[7]

X. Bi and S. Zhai. 2016. Predicting Finger-Touch Accuracy Based on the Dual Gaussian Distribution Model. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology(UIST ’16). ACM, New York, NY, USA, 313–319. https://doi.org/10.1145/2984511.2984546

Digital Library

[8]

S. M. Billah, Y.-J. Ko, 2019. Accessible gesture typing for non-visual text entry on smartphones. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[9]

R. Blanch and M. Ortega. 2011. Benchmarking pointing techniques with distractors: adding a density factor to Fitts’ pointing paradigm. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1629–1638.

Digital Library

[10]

BlindSquare. 2020. BlindSquare: Pioneering accessible navigation – indoors and outdoors. https://www.blindsquare.com/

[11]

K. P. Burnham and D. R. Anderson. A practical information-theoretic approach. Model selection and multimodel inference 2 (2002).

[12]

E. Crossman and P. Goodeve. Feedback control of hand-movement and Fitts’ Law. The Quarterly journal of experimental psychology. A, Human experimental psychology 35 (06 1968/1983), 251–78. https://doi.org/10.1080/14640748308402133

[13]

A. D. Edwards. Modelling blind users’ interactions with an auditory computer interface. International journal of man-Machine studies 30, 5 (1989), 575–589.

[14]

M. El Lahib, J. Tekli, Evaluating Fitts’ law on vibrating touch-screen to improve visual data accessibility for blind users. International Journal of Human-Computer Studies 112 (2018), 16–27.

[15]

P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement.Journal of experimental psychology 47, 6 (1954), 381.

[16]

P. M. Fitts and J. R. Peterson. Information capacity of discrete motor responses.Journal of experimental psychology 67, 2 (1964), 103.

[17]

K. Gajos and D. S. Weld. 2004. SUPPLE: Automatically Generating User Interfaces. In Proceedings of the 9th International Conference on Intelligent User Interfaces(IUI ’04). Association for Computing Machinery, New York, NY, USA, 93–100. https://doi.org/10.1145/964442.964461

Digital Library

[18]

K. Z. Gajos, J. J. Long, 2006. Automatically Generating Custom User Interfaces for Users with Physical Disabilities. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility(Assets ’06). Association for Computing Machinery, New York, NY, USA, 243–244. https://doi.org/10.1145/1168987.1169036

Digital Library

[19]

K.-C. Gan and E. R. Hoffmann. Geometrical conditions for ballistic and visually controlled movements. Ergonomics 31, 5 (1988), 829–839.

[20]

K. Goldberg, S. Faridani, A new derivation and dataset for Fitts’ law of human motion. EECS Dept., Univ. California, Berkeley, CA, USA, Tech. Rep. UCB/EECS-2013-171 (2013).

[21]

T. Guerreiro, K. Montague, 2015. Blind people interacting with large touch surfaces: Strategies for one-handed and two-handed exploration. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces. 25–34.

Digital Library

[22]

L. He, R. Wang, 2020. PneuFetch: Supporting Blind and Visually Impaired People to Fetch Nearby Objects via Light Haptic Cues. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems(CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3334480.3383095

Digital Library

[23]

G. Hinton, l. Deng, Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups. Signal Processing Magazine, IEEE 29 (11 2012), 82–97. https://doi.org/10.1109/MSP.2012.2205597

[24]

C. Holz and P. Baudisch. 2010. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’10). ACM, New York, NY, USA, 581–590. https://doi.org/10.1145/1753326.1753413

Digital Library

[25]

C. Holz and P. Baudisch. 2011. Understanding Touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). ACM, New York, NY, USA, 2501–2510. https://doi.org/10.1145/1978942.1979308

[26]

J. Hong, A. Pradhan, 2017. Evaluating Wrist-Based Haptic Feedback for Non-Visual Target Finding and Path Tracing on a 2D Surface. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’17). Association for Computing Machinery, New York, NY, USA, 210–219. https://doi.org/10.1145/3132525.3132538

Digital Library

[27]

S. K. Kane, J. P. Bigham, Fully accessible touch screens for the blind and visually impaired. University of Washington(2011).

[28]

S. K. Kane, J. P. Bigham, 2008. Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility. ACM, 73–80.

Digital Library

[29]

S. K. Kane, M. R. Morris, 2011. Access Overlays: Improving Non-Visual Access to Large Touch Screens for Blind Users. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology(UIST ’11). Association for Computing Machinery, New York, NY, USA, 273–282. https://doi.org/10.1145/2047196.2047232

Digital Library

[30]

S. Keele. Movement Control in Skilled Motor Performance. Psychological Bulletin 70 (12 1968). https://doi.org/10.1037/h0026739

[31]

G. LLC.2020. Accessibility in Material Design. https://material.io/design/usability/accessibility.html.

[32]

I. S. MacKenzie. A note on the information-theoretic basis for Fitts’ law. Journal of motor behavior 21, 3 (1989), 323–330.

[33]

I. S. MacKenzie. Fitts’ law as a research and design tool in human-computer interaction. Human-computer interaction 7, 1 (1992), 91–139.

Digital Library

[34]

D. E. Meyer, R. A. Abrams, Optimality in human motor performance: ideal control of rapid aimed movements.Psychological review 95, 3 (1988), 340.

[35]

D. E. Meyer, J. E. Keith-Smith, 1990. Speed-accuracy tradeoffs in aimed movements: Toward a theory of rapid voluntary action.

[36]

J. Müller, A. Oulasvirta, Control theoretic models of pointing. ACM Transactions on Computer-Human Interaction (TOCHI) 24, 4(2017), 1–36.

[37]

A. Oulasvirta, N. R. Dayama, Combinatorial optimization of graphical user interface designs. Proc. IEEE 108, 3 (2020), 434–464.

[38]

M. Ptito, S. M. Moesgaard, Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain 128, 3 (2005), 606–614.

[39]

E. Ricciardi, D. Bonino, Do we really need vision? How blind people “see” the actions of others. Journal of Neuroscience 29, 31 (2009), 9719–9724.

[40]

S. Sarcar, J. P. Jokinen, Ability-based optimization of touchscreen interactions. IEEE Pervasive Computing 17, 1 (2018), 15–26.

Digital Library

[41]

A. Savitzky and M. J. Golay. Smoothing and differentiation of data by simplified least squares procedures.Analytical chemistry 36, 8 (1964), 1627–1639.

[42]

TalkBack. 2020. TalkBack: An Open Source Screenreader For Android. https://support.google.com/accessibility/android/answer/6283677

[43]

S. Watanabe. Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research 11, Dec (2010), 3571–3594.

[44]

A. T. Welford. The measurement of sensory-motor performance: Survey and reappraisal of twelve years’ progress. Ergonomics 3, 3 (1960), 189–230.

[45]

A. T. Welford. 1968. Fundamentals of skill. Methuen, New York, NY, USA.

Cited By

Kodandaram SUckun UBi XRamakrishnan IAshok V(2024)Enabling Uniform Computer Interaction Experience for Blind Users through Large Language ModelsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675605(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675605

Recommendations

Accessible smartphones for blind users: A case study for a wayfinding system

While progress on assistive technologies have been made, some blind users still face several problems opening and using basic functionalities when interacting with touch interfaces. Sometimes, people with visual impairments may also have problems ...
Making Tabletop Interaction Accessible for Blind Users
ITS '14: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces

Tabletop systems and their interaction capabilities are typically a domain for sighted people only. While the content on the tabletop can already be made accessible to blind people, the interaction above the tabletop is still inaccessible. This paper ...
Making touch-based kiosks accessible to blind users through simple gestures

Touch-based interaction is becoming increasingly popular and is commonly used as the main interaction paradigm for self-service kiosks in public spaces. Touch-based interaction is known to be visually intensive, and current non-haptic touch-display ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction

September 2021

637 pages

ISBN:9781450383288

DOI:10.1145/3447526

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)
Finnish Center for Artificial Intelligence FCAI
NIH (National Institutes of Health)
The Academy of Finland projects Human Automata
The Academy of Finland projects Human BAD

Conference

MobileHCI '21

Sponsor:

SIGCHI

MobileHCI '21: 23rd International Conference on Mobile Human-Computer Interaction

September 27 - October 1, 2021

Toulouse & Virtual, France

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
395
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)20

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kodandaram SUckun UBi XRamakrishnan IAshok V(2024)Enabling Uniform Computer Interaction Experience for Blind Users through Large Language ModelsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675605(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675605

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten