research-article

Access lens: a gesture-based screen reader for real-world documents

Authors:

Shaun K. Kane,

Brian Frey,

Jacob O. WobbrockAuthors Info & Claims

CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 347 - 350

https://doi.org/10.1145/2470654.2470704

Published: 27 April 2013 Publication History

Get Access

Abstract

Gesture-based touch screen user interfaces, when designed to be accessible to blind users, can be an effective mode of interaction for those users. However, current accessible touch screen interaction techniques suffer from one serious limitation: they are only usable on devices that have been explicitly designed to support them. Access Lens is a new interaction method that uses computer vision-based gesture tracking to enable blind people to use accessible gestures on paper documents and other physical objects, such as product packages, device screens, and home appliances. This paper describes the development of Access Lens hardware and software, the iterative design of Access Lens in collaboration with blind computer users, and opportunities for future development.

Supplementary Material

suppl.mov (chi2165-file3.mp4)

Supplemental video

Download
6.53 MB

References

[1]

Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tararowicz, A. White, B., White, S., and Yeh, T. VizWiz: nearly real-time answers to visual questions. Proc. UIST '10, (2010), 333--342.

Digital Library

Google Scholar

[2]

Bradski, G. The OpenCV library. Doctor Dobbs Journal 25, 11 (2000), 120--126.

Google Scholar

[3]

Kane, S.K., Avrahami, D., Wobbrock, J.O., Harrison, B., Rea, A.D., Philipose, M., and LaMarca, A. Bonfire: a nomadic system for hybrid laptop-tabletop interaction. Proc. UIST '09, (2009), 129--138.

Digital Library

Google Scholar

[4]

Kane, S.K., Bigham, J.P., and Wobbrock, J.O. Slide Rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques. Proc. ASSETS '08, (2008), 73--80.

Digital Library

Google Scholar

[5]

Kane, S.K., Morris, M.R., Perkins, A.Z., Wigdor, D., Ladner, R.E., and Wobbrock, J.O. Access Overlays: improving non-visual access to large touch screens for blind users. Proc. UIST '11, (2011), 273--280.

Digital Library

Google Scholar

[6]

Levenshtein, V.I. Binary codes capable of correcting spurious insertions and deletions of ones. Problems of Information Transmission 1, 1 (1965), 8--17.

Google Scholar

[7]

Mahmoud, T.M. A new fast skin color detection technique. WEAST 43, (2008), 501--505.

Google Scholar

[8]

Nanayakkara, S., Shilkrot, R., and Maes, P. EyeRing: a finger-worn assistant. Proc. CHI EA '12, (2012), 1961-- 1966.

Digital Library

Google Scholar

[9]

Stauffer, C. and Grimson, W.E.L. Adaptive background mixture models for real-time tracking. Proc. CVPR '00, 252--258.

Google Scholar

[10]

Sukthankar, R., Stockton, R.G., and Mullin, M.D. Smarter presentations: exploiting homography in camera-projector systems. Proc. ICCV '01, 247--253.

Google Scholar

[11]

Wellner, P. Interacting with paper on the DigitalDesk. Communications of the ACM 36, 7 (1993), 87--96.

Digital Library

Google Scholar

[12]

White, S., Ji, H., and Bigham, J.P. EasySnap: real-time audio feedback for blind photography. Proc. UIST '10, (2010), 409--410.

Digital Library

Google Scholar

[13]

Wong, E.K. and Chen, M. A new robust algorithm for video text extraction. Pattern Recognition 36, 6 (2003), 1397--1406.

Crossref

Google Scholar

Cited By

View all

Xie JYu RZhang HLee SBillah SCarroll J(2024)BubbleCam: Engaging Privacy in Remote Sighted AssistanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642030(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642030
Hasegawa RIchino J(2023)Inaccessibility maps to support sighted people using visually impaired people’s white cane dataTechnology and Disability10.3233/TAD-23001435:4(225-244)Online publication date: 27-Dec-2023
https://doi.org/10.3233/TAD-230014
Dang KKorreshi HIqbal YLee S(2023)Opportunities for Accessible Virtual Reality Design for Immersive Musical Performances for Blind and Low-Vision PeopleProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3614540(1-21)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3614540
Show More Cited By

Index Terms

Access lens: a gesture-based screen reader for real-world documents
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Braille Touch: mobile touchscreen text entry for the visually impaired
MobileHCI '12: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services companion

We present a demonstration of BrailleTouch, an accessible keyboard for blind users on a touchscreen smartphone (see Figure 1). Based on the standard Perkins Brailler, BrailleTouch implements a six-key chorded braille soft keyboard [1]. We will briefly ...
An evaluation of BrailleTouch: mobile touchscreen text entry for the visually impaired
MobileHCI '12: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services

We present the evaluation of BrailleTouch, an accessible keyboard for blind users on touchscreen smartphones. Based on the standard Perkins Brailler, BrailleTouch implements a six-key chorded braille soft keyboard. Eleven blind participants typed for ...
Access overlays: improving non-visual access to large touch screens for blind users
UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

Many touch screens remain inaccessible to blind users, and those approaches to providing access that do exist offer minimal support for interacting with large touch screens or spatial data. In this paper, we introduce a set of three software-based ...

Comments

Information & Contributors

Information

Published In

CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2013

3550 pages

ISBN:9781450318990

DOI:10.1145/2470654

General Chair:
Wendy E. Mackay
INRIA
,
Program Chairs:
Stephen Brewster
Glasgow University
,
Susanne Bødker
University of Aarhus

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '13

Sponsor:

SIGCHI

CHI '13: CHI Conference on Human Factors in Computing Systems

April 27 - May 2, 2013

Paris, France

Acceptance Rates

CHI '13 Paper Acceptance Rate 392 of 1,963 submissions, 20%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

52
Total Citations
View Citations
874
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)10

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Xie JYu RZhang HLee SBillah SCarroll J(2024)BubbleCam: Engaging Privacy in Remote Sighted AssistanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642030(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642030
Hasegawa RIchino J(2023)Inaccessibility maps to support sighted people using visually impaired people’s white cane dataTechnology and Disability10.3233/TAD-23001435:4(225-244)Online publication date: 27-Dec-2023
https://doi.org/10.3233/TAD-230014
Dang KKorreshi HIqbal YLee S(2023)Opportunities for Accessible Virtual Reality Design for Immersive Musical Performances for Blind and Low-Vision PeopleProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3614540(1-21)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3614540
Wang XKayukawa STakagi HAsakawa C(2023)TouchPilot: Designing a Guidance System that Assists Blind People in Learning Complex 3D StructuresProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608426(1-18)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608426
Otterbacher J(2023)Computer Vision, Human Likeness, and Problematic BehaviorsAdjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3563359.3597381(206-214)Online publication date: 26-Jun-2023
https://dl.acm.org/doi/10.1145/3563359.3597381
Li JYan ZShah ALazar JPeng H(2023)Toucha11y: Making Inaccessible Public Touchscreens AccessibleProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581254(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581254
Dudley JYin LGaraj VKristensson P(2023)Inclusive Immersion: a review of efforts to improve accessibility in virtual reality, augmented reality and the metaverseVirtual Reality10.1007/s10055-023-00850-827:4(2989-3020)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10055-023-00850-8
Kong JZhong MFogarty JWobbrock J(2022)Quantifying Touch: New Metrics for Characterizing What Happens During a TouchProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544804(1-13)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544804
Lee SAldas NLee CRosson MCarroll JNarayanan V(2022)AIGuide: Augmented Reality Hand Guidance in a Visual ProstheticACM Transactions on Accessible Computing10.1145/350850115:2(1-32)Online publication date: 19-May-2022
https://dl.acm.org/doi/10.1145/3508501
Xie JReddie MLee SBillah SZhou ZTsai CCarroll J(2022)Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted AssistanceACM Transactions on Computer-Human Interaction10.1145/350129829:4(1-40)Online publication date: 31-Mar-2022
https://dl.acm.org/doi/10.1145/3501298
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Braille Touch: mobile touchscreen text entry for the visually impaired

An evaluation of BrailleTouch: mobile touchscreen text entry for the visually impaired

Access overlays: improving non-visual access to large touch screens for blind users