research-article

Tactile Graphics with a Voice

Authors:

Catherine M. Baker,

Lauren R. Milne,

Jeffrey Scofield,

Cynthia L. Bennett,

Richard E. LadnerAuthors Info & Claims

ACM Transactions on Accessible Computing (TACCESS), Volume 8, Issue 1

Article No.: 3, Pages 1 - 22

https://doi.org/10.1145/2854005

Published: 13 January 2016 Publication History

Abstract

We discuss the development of Tactile Graphics with a Voice (TGV), a system used to access label information in tactile graphics using QR codes. Blind students often rely on tactile graphics to access textbook images. Many textbook images have a large number of text labels that need to be made accessible. In order to do so, we propose TGV, which uses QR codes to replace the text, as an alternative to Braille. The codes are read with a smartphone application. We evaluated the system with a longitudinal study where 10 blind and low-vision participants completed tasks using three different modes on the smartphone application: (1) no guidance, (2) verbal guidance, and (3) finger-pointing guidance. Our results show that TGV is an effective way to access text in tactile graphics, especially for those blind users who are not fluent in Braille. We also found that preferences varied greatly across the modes, indicating that future work should support multiple modes. We expand upon the algorithms we used to implement the finger pointing, algorithms to automatically place QR codes on documents. We also discuss work we have started on creating a Google Glass version of the application.

References

[1]

Frances K., Aldrich and Linda Sheppard. 2001. Tactile graphics in school education: Perspectives from pupils. British Journal of Visual Impairment, 19, 2, 69--73.

[2]

Catherine M. Baker, Lauren R. Milne, Jeffrey Scofield, Cynthia L. Bennett, and Richard E. Ladner. 2014. Tactile graphics with a voice: Using QR codes to access text in tactile graphics. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS'14), 75--82.

Digital Library

[3]

Jeffrey P. Bigham, Chandrika Jayant, Andrew Miller, Brandyn White, and Tom Yeh. 2010. VizWiz::LocateIt - Enabling blind people to locate objects in their environment. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops (CVPRW'10), 65--72.

[4]

Braille Cell Dimensions. 2009. http://www.tiresias.org/research/reports/braille_cell.htm.

[5]

Ahmed Elgammal, Crystal Muang, and Dunxu Hu. 2009. Skin detection-a short tutorial. Encyclopedia of Biometrics, 1--10. Retrieved from http://pdf.aminer.org/000/312/166/finding_facial_features_using_an_hls_colour_space.pdf.

[6]

Daniel Goransson. 2011. New VoiceOver Features in iOS 5. Retrieved from http://axslab.com/articles/new-voiceover-features-in-ios5.

[7]

Berchie Woods Gordon-Holliday, L. E. Yunker, G. Vannatta, and F. J. Crosswhite. 1999. Advanced Mathematical Concepts: Precalculus with Applications. Glencoe/McGraw-Hill.

[8]

Bill Holton. 2013. Voiceye: A breakthrough in document access. AFB AccessWorld Magazine.

[9]

Chandrika Jayant, Hanjie Ji, Samuel White, and Jeffrey P. Bigham. 2011. Supporting blind photography. In Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS'11), 203--210.

Digital Library

[10]

Chandrika Jayant, Matt Renzelmann, Dana Wen, Satria Krisnandi, Richard Ladner, and Dan Comden. 2007. Automated tactile graphics translation. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS'07), 75.

Digital Library

[11]

Shaun K. Kane, Brian Frey, and Jacob O. Wobbrock. 2013. Access Lens: A gesture-based screen reader for real-world documents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13), 347--350.

Digital Library

[12]

Steven Landau, Steven Holborow, and Jane Erin. 2004. The use of the talking tactile tablet for delivery of standardized tests. In Proceedings of the Annual International Technology and Persons with Disabilities Conference.

[13]

Steven Landau and Joshua Miele. 2010. Talking Tactile Apps for the Pulse Pen: STEM Binder. Retrieved from https://docs.google.com/presentation/d/1Ylscpk6QKX7Y5WCW3CFnhZDJJlL4X5ki8gHdcJvBp70/present#slide=id.i0.

[14]

Suranga Nanayakkara, Roy Shilkrot, Kian Peen Yeo, and Pattie Maes. 2013. EyeRing: A finger-worn input device for seamless interactions with our surroundings. In Proceedings of the 4th Augmented Human International Conference on AH'13. 13--20.

Digital Library

[15]

National Federation of the Blind Jernigan Institute. 2009. The Braille Literacy Crisis in America: Facing the Truth, Reversing the Trend, Empowering the Blind. Retrieved from https://www.nfb.org.

[16]

Nemeth Braille. http://www.braillebug.org/nemeth_braille.asp.

[17]

Son Lam Phung, Abdesselam Bouzerdoum, and Douglas Chai. 2003. Skin segmentation using color and edge information. In Proceedings of the 7th International Symposium on Signal Processing and Its Applications (ISSPA'03), 1 (July), 525--528.

[18]

Son Lam Phung, Abdesselam Bouzerdoum, and Douglas Chai. 2005. Skin segmentation using color pixel classification: Analysis and comparison. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1, 148--154.

Digital Library

[19]

2011. QR Code Minimum Size. http://www.qrstuff.com/blog/2011/11/23/qr-code-minimum-size.

[20]

Audrey C. Rule, Greg P. Stefanich, Robert M. Boody, and Belinda Peiffer. 2011. Impact of adaptive materials on teachers and their students with visual impairments in secondary science and mathematics classes. International Journal of Science Education 33, January 2015, 865--887. http://doi.org/10.1080/09500693.2010.506619

[21]

Stuart Russell and Peter Norvig. 2009. Artificial Intelligence: A Modern Approach, 3rd ed.

Digital Library

[22]

Linda Sheppard and Frances K. Aldrich. 2001. Tactile graphics in school education: Perspectives from teachers. British Journal of Visual Impairment, 19, 3, 93--97. http://doi.org/10.1177/026461960101900303

[23]

Richard Szeliski. 2010. Computer vision: Algorithms and applications. Computer 5, 832.

Digital Library

[24]

Ender Tekin and James M. Coughlan. 2010. A mobile phone application enabling visually impaired users to find and read product barcodes. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6180 LNCS (Part 2), 290--295.

[25]

Marynel Vázquez and Aaron Steinfeld. 2012. Helping visually impaired users properly aim a camera. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility, 95--102. http://doi.org/10.1145/2384916.2384934

Digital Library

Cited By

Jiang CKuang EFan M(2025)How Can Haptic Feedback Assist People with Blind and Low Vision (BLV): A Systematic Literature ReviewACM Transactions on Accessible Computing10.1145/371193118:1(1-57)Online publication date: 13-Jan-2025
https://dl.acm.org/doi/10.1145/3711931
Vargas PTrevisan D(2024)Revisiting visual accessibility with non-textual content: challenges and solutions for human-computer interactionProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702105(1-18)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702105
Poddar RNaik TPunnam MChourasia KPandurangan RPaali RBhat NBiradar BDeshpande VGhosh DChaudhuri SChakraborty DPrakash ASwaminathan M(2024)Experiences from Running a Voice-Based Education Platform for Children and Teachers with Visual ImpairmentsACM Journal on Computing and Sustainable Societies10.1145/36773232:3(1-35)Online publication date: 8-Aug-2024
https://dl.acm.org/doi/10.1145/3677323
Show More Cited By

Index Terms

Tactile Graphics with a Voice
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Understanding what you feel: A Mobile Audio-Tactile System for Graphics Used at Schools with Students with Visual Impairment
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

A lot of information is nowadays presented graphically. However, students with blindness do not have access to visual information. Providing an alternative text is not always the appropriate solution as exploring graphics to discover information ...
Visually Augmented Audio-Tactile Graphics for Visually Impaired People
Paper from ASSETS 2016 and Regular Papers

Tactile graphics play an essential role in knowledge transfer for blind people. The tactile exploration of these graphics is often challenging because of the cognitive load caused by physiological constraints and their complexity. The coupling of ...
Tactile graphics with a voice: using QR codes to access text in tactile graphics
ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility

Textbook figures are often converted into a tactile format for access by blind students. These figures are not truly accessible unless the text within the figures is also made accessible. A common solution to access text in a tactile image is to use ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Accessible Computing

ACM Transactions on Accessible Computing Volume 8, Issue 1

Special Issue (Part 1) of Papers from ASSETS 2014

January 2016

82 pages

ISSN:1936-7228

EISSN:1936-7236

DOI:10.1145/2878808

Editors:
Matt Huenerfauth
Rochester Institute of Technology, USA
,
Kathleen F. McCoy
University of Delaware, USA

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2016

Accepted: 01 October 2015

Revised: 01 October 2015

Received: 01 April 2015

Published in TACCESS Volume 8, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

U.S. Department of Education, Office of Special Education Programs (Cooperative Agreement #H327B100001)
National Science Foundation
National Science Foundation Graduate Research Fellowship
U.S. Department of Education or the National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
493
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)8

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang CKuang EFan M(2025)How Can Haptic Feedback Assist People with Blind and Low Vision (BLV): A Systematic Literature ReviewACM Transactions on Accessible Computing10.1145/371193118:1(1-57)Online publication date: 13-Jan-2025
https://dl.acm.org/doi/10.1145/3711931
Vargas PTrevisan D(2024)Revisiting visual accessibility with non-textual content: challenges and solutions for human-computer interactionProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702105(1-18)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702105
Poddar RNaik TPunnam MChourasia KPandurangan RPaali RBhat NBiradar BDeshpande VGhosh DChaudhuri SChakraborty DPrakash ASwaminathan M(2024)Experiences from Running a Voice-Based Education Platform for Children and Teachers with Visual ImpairmentsACM Journal on Computing and Sustainable Societies10.1145/36773232:3(1-35)Online publication date: 8-Aug-2024
https://dl.acm.org/doi/10.1145/3677323
Zong JPedraza Pineros IChen MHajas DSatyanarayan A(2024)Umwelt: Accessible Structured Editing of Multi-Modal Data RepresentationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641996(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641996
Robinson Moore WKalal MTennison JGiudice NGorlewicz J(2024)Spatial Audio-Enhanced Multimodal Graph Rendering for Efficient Data Trend Learning on Touchscreen DevicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641959(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641959
Hou WRiccò D(2024)Accessible Design for Museums: A Systematic Review on Multisensory Experience Based on Digital TechnologyAdvances in Design and Digital Communication V10.1007/978-3-031-77566-6_21(291-305)Online publication date: 24-Dec-2024
https://doi.org/10.1007/978-3-031-77566-6_21
Maćkowski MKawulok MBrzoza PJanczy MSpinczyk D(2023)An Alternative Audio-Tactile Method of Presenting Structural Information Contained in Mathematical Drawings Adapted to the Needs of the BlindApplied Sciences10.3390/app1317998913:17(9989)Online publication date: 4-Sep-2023
https://doi.org/10.3390/app13179989
Maćkowski MBrzoza PKawulok MMeisel RSpinczyk D(2023)Multimodal Presentation of Interactive Audio-Tactile Graphics Supporting the Perception of Visual Information by Blind PeopleACM Transactions on Multimedia Computing, Communications, and Applications10.1145/358607619:5s(1-22)Online publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1145/3586076
Maćkowski MBrzoza P(2022)Accessible Tutoring Platform Using Audio-Tactile Graphics Adapted for Visually Impaired PeopleSensors10.3390/s2222875322:22(8753)Online publication date: 12-Nov-2022
https://doi.org/10.3390/s22228753
Elavsky FBennett CMoritz D(2022)How accessible is my visualization? Evaluating visualization accessibility with ChartabilityComputer Graphics Forum10.1111/cgf.1452241:3(57-70)Online publication date: 12-Aug-2022
https://doi.org/10.1111/cgf.14522
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents