research-article

Understanding what you feel: A Mobile Audio-Tactile System for Graphics Used at Schools with Students with Visual Impairment

Authors:

Giuseppe Melfi,

Thorsten Schwarz,

Gerhard Jaworek,

Rainer StiefelhagenAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 12

https://doi.org/10.1145/3313831.3376508

Published: 23 April 2020 Publication History

Abstract

A lot of information is nowadays presented graphically. However, students with blindness do not have access to visual information. Providing an alternative text is not always the appropriate solution as exploring graphics to discover information independently is a fundamental part of the learning process. In this work, we introduce a mobile audio-tactile learning environment, which facilitates the incorporation of real educational material. We evaluate our system by comparing three methods of interaction with tactile graphics: A tactile graphic augmented by (1) a document with key index information in Braille, (2) a digital document with key index information and (3) the TPad system, an audio-tactile solution meeting the specific needs within the school context. Our study shows that the TPad system is suitable for educational environments. Moreover, compared to the other methods TPad is faster to explore tactile graphics and it suggests a promising effect on the memorization of information.

Supplementary Material

MP4 File (a381-melfi-presentation.mp4)

Download
42.93 MB

References

[1]

Frances K. Aldrich, Linda Sheppard and Yvonne Hindle. 2002. First steps towards a model of tactile graphicacy. The British Journal of Visual Impairment, 2002, 20:2, 62--67

[2]

Frances K. Aldrich and Linda Sheppard. 2001. Tactile graphics in school education: perspectives from pupils. The British Journal of Visual Impairment, 2001, 19:2, 69--73

[3]

Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang. 2018. Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering. 60776086. 10.1109/CVPR.2018.00636

[4]

C. M. Baker, L. R. Milne, J. Scofield, C. L. Bennett, and R. E. Ladner. Tactile graphics with a voice: using QR codes to access text in tactile graphics. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS '14, pages 75--82. ACM, 2014.

Digital Library

[5]

Anke M. Brock, Christophe Jouffrais. 2015. Interactive audio-tactile maps for visually impaired people. ACM SIGACCESS Accessibility and Computing (ACM Digital Library). Association for Computing Machinery (ACM). 3--12.

[6]

Anke M. Brock, Philippe Truillet, Bernard Oriola, Delphine Picard and Christophe Jouffrais. 2015. Interactivity Improves Usability of Geographic Maps for Visually Impaired People. Human-Computer Interaction. 30:2, 156--194.

Digital Library

[7]

Julie Ducasse, Anke Brock and Christophe Jouffrais. 2017. Accessible Interactive Maps for Visually Impaired Users.

[8]

Giovanni Fusco and Valerie S. Morash. 2015. The Tactile Graphics Helper: Providing Audio Clarification for Tactile Graphics Using Machine Vision. ASSETS 2015, 97--106

Digital Library

[9]

J. A. Gardner and V. Bulatov. Scientific diagrams made easy with IVEO TM. In Computers Helping People with Special Needs, pages 1243--1250. Springer, 2006.

Digital Library

[10]

Stéphanie Giraud, Anke M. Brock, Marc J.-M. Macé and Christophe Jouffrais. 2017. Map Learning with a 3D Printed Interactive Small-Scale Model: Improvement of Space and Text Memorization in Visually Impaired Students. In Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2017.00930

[11]

Timo Götzelmann. 2018. Visually Augmented AudioTactile Graphics for Visually Impaired People. ACM Transactions on Accessible Computing. 11. 1--31. 10.1145/3186894.

[12]

Mihail Ivanchev, Francis Zinke and Ulrike Lucke. 2014. Pre-journey Visualization of Travel Routes for the Blind on Refreshable Interactive Tactile Displays. ICCHP 2014, Part II, LNCS 8548, 81--88.

[13]

Chandrika Jayant, Matthew Renzelmann, Dana Wen, Satria Krisnandi, R.E. Ladner and Dan Comden. 2007. Automated Tactile Graphics Translation: In the Field. 75--82. 10.1145/1296843.1296858.

[14]

Ernst Klett. 2010. Prisma Biologie Berufsfachschule 9./10. Schuljahr. ISBN: 978--3--12-068450--3

[15]

S. Landau, G. Bourquin, J. Miele, and A. Van Schaack. Demonstration of a universally accessible audio-haptic transit map built on a digital pen-based platform. In Proceedings of the 3rd International Workshop on Haptic and Audio Interaction Design, pages 23--24, 2008.

[16]

S. Landau, R. Holborow and E. Jane. The Use of the Talking Tactile Tablet for Delivery of Standardized Tests. In Proc. CSUN 2004

[17]

Steven Landau and Lesley Wells. 2003. Merging tactile sensory input and audio data by means of the talking tactile tablet. In Proceedings of the European Conference on Haptics (EuroHaptics'03). 414--418.

[18]

K. Minatani. 2014. A Proposal for an Automated Method to Produce Embossed Graphics for Blind Persons. In: Stephanidis C., Antona M. (eds) Universal Access in Human-Computer Interaction. Universal Access to Information and Knowledge. UAHCI 2014. Lecture Notes in Computer Science, vol 8514. Springer, Cham

[19]

Liam O'Sullivan, Lorenzo Picinali, Andrea Gerino and Douglas Cawthrone. 2015. A Prototye Audio-Tactile Map System with an Advanced Auditory Display. International Journal of Mobile Human Computer Interaction, 7(4), 53--75

Digital Library

[20]

Grégory Petit, Aude Dufresne, Vincent Levesque, Vincent Hayward and Nicole Trudeau. 2008. Refreshable Tactile Graphics Applied to Schoolbook Illustrations for Students with Visual Impairment. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility. 89--96.

Digital Library

[21]

D. Prescher, J. Bornschein and G. Weber. 2014. Production of Accessible Tactile Graphics. In: Miesenberger K., Fels D., Archambault D., Pe?áz P., Zagler W. (eds) Computers Helping People with Special Needs. ICCHP 2014. Lecture Notes in Computer Science, vol 8548. Springer, Cham

[22]

Andreas Reichinger, Anton Fuhrmann, Stefan Maierhofer and Werner Purgathofer. 2016. GestureBased Interactive Audio Guide on Tactile Reliefs. 10.1145/2982142.2982176.

[23]

Claus Reinhardt. 2012. Biologie Natura - Biologie für berufliche Gymnasien, berufliche Oberstufe. ISBN: 978--3--12-045306--2

[24]

L. Penny Rosenblum, Li Cheng, and Carole R. Beal. 2018. Teachers of Students with Visual Impairments Share Experiences and Advice for Supporting Students in Understanding Graphics. Journal of Visual Impairment & Blindness. October 2018, 475--487.

[25]

Linda Sheppard and Frances K. Aldrich. 2001. Tactile graphics in school education: perspectives from teachers. The British Journal of Visual Impairment, 2001 19:3, 93--97.

[26]

Sarit Szpiro, Shafeka Hashash, Yuhang Zhao and Shiri Azenkot. 2016. How People with Low Vision Access Computing Devices: Understanding Challenges and Opportunities. 171--180. 10.1145/2982142.2982168.

[27]

Lauren Thevin and Anke Brock. 2018). Augmented Reality for People with Visual Impairments: Designing and Creating Audio-Tactile Content from Existing Objects. 10.1007/978--3--319--94274--2_26.

[28]

Zheshen Wang, Baoxin Li, Terri Hedgpeth and Teresa Haven. 2009. Instant Tactiled-Audio Map: Enabling Access to Digital Maps for People with Visual Impairment. ASSETS, 43--50

[29]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Cho Kyunghyun, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel and Y. Bengio. 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention.

[30]

Kim T. Zebehazy and Adam P. Wilton. 2014. Quality, importance, and instruction: the perspectives of teachers of students with visual impairments on graphics use by students. Journal of Visual Impairment & Blindness, July-August 2014, 275--286

[31]

Limin Zeng and Gerhard Weber. 2011. Accessible Maps for the Visually Impaired. CEUR Workshop Proceedings. 792.

Cited By

Fan DKim GTomassetti OPatel SO'Modhrain SLee VFollmer S(2024)Tangible Stats: An Embodied and Multimodal Platform for Teaching Data and Statistics to Blind and Low Vision StudentsExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650793(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650793
Jiang CFan YXie JKuang EZhang KFan M(2024)Designing Unobtrusive Modulated Electrotactile Feedback on Fingertip Edge to Assist Blind and Low Vision (BLV) People in Comprehending ChartsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642546(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642546
Hutson JHutson PHutson JHutson P(2024)Inclusivity & EnvironmentInclusive Smart Museums10.1007/978-3-031-43615-4_3(85-125)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-43615-4_3
Show More Cited By

Index Terms

Understanding what you feel: A Mobile Audio-Tactile System for Graphics Used at Schools with Students with Visual Impairment

Recommendations

Visually Augmented Audio-Tactile Graphics for Visually Impaired People
Paper from ASSETS 2016 and Regular Papers

Tactile graphics play an essential role in knowledge transfer for blind people. The tactile exploration of these graphics is often challenging because of the cognitive load caused by physiological constraints and their complexity. The coupling of ...
Tactile graphics with a voice: using QR codes to access text in tactile graphics
ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility

Textbook figures are often converted into a tactile format for access by blind students. These figures are not truly accessible unless the text within the figures is also made accessible. A common solution to access text in a tactile image is to use ...
Tactile graphics with a voice demonstration
ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility

Textbook images are converted into tactile graphics to be made accessible to blind and low vision students. The text labels on these graphics are an important part of the image and must be made accessible as well. The graphics usually have the labels ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
727
Total Downloads

Downloads (Last 12 months)155
Downloads (Last 6 weeks)9

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fan DKim GTomassetti OPatel SO'Modhrain SLee VFollmer S(2024)Tangible Stats: An Embodied and Multimodal Platform for Teaching Data and Statistics to Blind and Low Vision StudentsExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650793(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650793
Jiang CFan YXie JKuang EZhang KFan M(2024)Designing Unobtrusive Modulated Electrotactile Feedback on Fingertip Edge to Assist Blind and Low Vision (BLV) People in Comprehending ChartsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642546(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642546
Hutson JHutson PHutson JHutson P(2024)Inclusivity & EnvironmentInclusive Smart Museums10.1007/978-3-031-43615-4_3(85-125)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-43615-4_3
Park JHwang GChoi UKim J(2023)Development of Digital Math Tactile Graphics for Students with Visual ImpairmentsJournal of Digital Contents Society10.9728/dcs.2023.24.9.196524:9(1965-1975)Online publication date: 30-Sep-2023
https://doi.org/10.9728/dcs.2023.24.9.1965
Maćkowski MKawulok MBrzoza PJanczy MSpinczyk D(2023)An Alternative Audio-Tactile Method of Presenting Structural Information Contained in Mathematical Drawings Adapted to the Needs of the BlindApplied Sciences10.3390/app1317998913:17(9989)Online publication date: 4-Sep-2023
https://doi.org/10.3390/app13179989
Moured OAlzalabny SSchwarz TRapp BStiefelhagen R(2023)Accessible Document Layout: An Interface for 2D Tactile DisplaysProceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3594806.3594811(265-271)Online publication date: 5-Jul-2023
https://dl.acm.org/doi/10.1145/3594806.3594811
Maćkowski MBrzoza PKawulok MMeisel RSpinczyk D(2023)Multimodal Presentation of Interactive Audio-Tactile Graphics Supporting the Perception of Visual Information by Blind PeopleACM Transactions on Multimedia Computing, Communications, and Applications10.1145/358607619:5s(1-22)Online publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1145/3586076
Cheng TZhang ZZong BZhao YChang ZKim YZheng CAbowd GOh H(2023)SwellSense: Creating 2.5D interactions with micro-capsule paperProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581125(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581125
Cole HRobinson A(2023)Thematic tactile maps for accessible flood mitigation planning: design and evaluationCartography and Geographic Information Science10.1080/15230406.2023.226474750:6(574-592)Online publication date: 31-Oct-2023
https://doi.org/10.1080/15230406.2023.2264747
Maćkowski MBrzoza PSpinczyk D(2023)An alternative method of audio-tactile presentation of graphical information in mathematics adapted to the needs of blindInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103122179:COnline publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103122
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents