research-article

Public Access

Facade: Auto-generating Tactile Interfaces to Appliances

Authors:

Xiang 'Anthony' Chen,

Scott E. Hudson,

Jennifer Mankoff,

Jeffrey P. BighamAuthors Info & Claims

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Pages 5826 - 5838

https://doi.org/10.1145/3025453.3025845

Published: 02 May 2017 Publication History

Abstract

Common appliances have shifted toward flat interface panels, making them inaccessible to blind people. Although blind people can label appliances with Braille stickers, doing so generally requires sighted assistance to identify the original functions and apply the labels. We introduce Facade - a crowdsourced fabrication pipeline to help blind people independently make physical interfaces accessible by adding a 3D printed augmentation of tactile buttons overlaying the original panel. Facade users capture a photo of the appliance with a readily available fiducial marker (a dollar bill) for recovering size information. This image is sent to multiple crowd workers, who work in parallel to quickly label and describe elements of the interface. Facade then generates a 3D model for a layer of tactile and pressable buttons that fits over the original controls. Finally, a home 3D printer or commercial service fabricates the layer, which is then aligned and attached to the interface by the blind person. We demonstrate the viability of Facade in a study with 11 blind participants.

Supplementary Material

ZIP File (pn2948-file4.zip)

Download
1.22 KB

suppl.mov (pn2948-file3.mp4)

Supplemental video

Download
88.03 MB

suppl.mov (pn2948p.mp4)

Supplemental video

Download
1.56 MB

References

[1]

American Foundation for Blind 2016. Braille: Deciphering the code. http://braillebug.afb.org/braille_deciphering.asp. (2016).

[2]

Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: Speeded up robust features. In Proceedings of the 9th European Conference on Computer Vision - Volume Part I (ECCV'06). Springer-Verlag, Berlin, Heidelberg, 404--417.

Digital Library

[3]

Jeffrey P. Bigham, Michael S. Bernstein, and Eytan Adar. 2015. Human-computer interaction and collective intelligence. Handbook of Collective Intelligence 57 (2015).

[4]

Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C. Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, and Tom Yeh. 2010a. VizWiz: Nearly real-time answers to visual questions. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, NY, NY, USA, 333--342.

Digital Library

[5]

Jeffrey P. Bigham, Chandrika Jayant, Andrew Miller, Brandyn White, and Tom Yeh. 2010b. VizWiz::LocateIt Enabling blind people to locate objects in their environment. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 65--72.

[6]

Erin Brady and Jeffrey P. Bigham. 2015. Crowdsourcing accessibility: Human-powered access technologies. Foundations and Trends in Human-Computer Interaction 8, 4 (2015), 273--372.

Digital Library

[7]

Erin Brady, Meredith Ringel Morris, Yu Zhong, Samuel White, and Jeffrey P. Bigham. 2013. Visual challenges in the everyday lives of blind people. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 2117--2126.

Digital Library

[8]

Stacy M. Branham and Shaun K. Kane. 2015. Collaborative accessibility: How blind and sighted companions co-create accessible home spaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 2373--2382.

Digital Library

[9]

Craig Brown and Amy Hurst. 2012. VizTouch: Automatically generated tactile visualizations of coordinate spaces. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI '12). ACM, NY, NY, USA, 131--138.

Digital Library

[10]

Emeline Brule, Gilles Bailly, Anke Brock, Frederic Valentin, Grégoire Denis, and Christophe Jouffrais. 2016. MapSense: Multi-sensory interactive maps for children living with visual impairments. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 445--457.

Digital Library

[11]

Erin Buehler, Stacy Branham, Abdullah Ali, Jeremy J. Chang, Megan Kelly Hofmann, Amy Hurst, and Shaun K. Kane. 2015. Sharing is caring: Assistive technology designs on thingiverse. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 525--534.

Digital Library

[12]

Erin Buehler, Amy Hurst, and Megan Hofmann. 2014a. Coming to grips: 3D printing for accessibility. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '14). ACM, NY, NY, USA, 291--292.

Digital Library

[13]

Erin Buehler, Shaun K. Kane, and Amy Hurst. 2014b. ABC and 3D: Opportunities and obstacles to 3D printing in special education environments. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '14). ACM, NY, NY, USA, 107--114.

Digital Library

[14]

H. Burton, D. G. McLaren, and R. J. Sinclair. 2006. Reading embossed capital letters: An fMRI study in blind and sighted individuals. Human brain mapping 27, 4 (2006), 325--339.

[15]

Xiang 'Anthony' Chen, Stelian Coros, Jennifer Mankoff, and Scott E. Hudson. 2015. Encore: 3D printed augmentation of everyday objects with printed-over, affixed and interlocked attachments. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, NY, NY, USA, 73--82.

Digital Library

[16]

Xiang 'Anthony' Chen, Jeeeun Kim, Jennifer Mankoff, Tovi Grossman, Stelian Coros, and Scott E. Hudson. 2016. Reprise: A design tool for specifying, generating, and customizing 3D printable adaptations on everyday objects. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 29--39.

Digital Library

[17]

Adrian A. de Freitas, Michael Nebeling, Xiang 'Anthony' Chen, Junrui Yang, Akshaye Shreenithi Kirupa Karthikeyan Ranithangam, and Anind K. Dey. 2016. Snap-To-It: A user-inspired platform for opportunistic device interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 5909--5920.

Digital Library

[18]

Martin A. Fischler and Robert C. Bolles. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6 (June 1981), 381--395.

Digital Library

[19]

Timo Götzelmann. 2016. LucentMaps: 3D printed audiovisual tactile maps for blind and visually impaired people. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 81--90.

Digital Library

[20]

Timo Götzelmann and Aleksander Pavkovic. 2014. Towards automatically generated tactile detail maps by 3D printers for blind persons. Springer International Publishing, Cham, 1--7.

[21]

Anhong Guo, Xiang 'Anthony' Chen, and Jeffrey P. Bigham. 2015. ApplianceReader: A wearable, crowdsourced, vision-based system to make appliances accessible. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, NY, NY, USA, 2043--2048.

Digital Library

[22]

Anhong Guo, Xiang 'Anthony' Chen, Haoran Qi, Samuel White, Suman Ghosh, Chieko Asakawa, and Jeffrey P. Bigham. 2016a. VizLens: A robust and interactive screen reader for interfaces in the real world. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 651--664.

Digital Library

[23]

Anhong Guo, Jeeeun Kim, Xiang 'Anthony' Chen, Tom Yeh, Scott E. Hudson, Jennifer Mankoff, and Jeffrey P. Bigham. 2016b. Facade: Auto-generating tactile interfaces to appliances. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 315--316.

Digital Library

[24]

Megan Hofmann, Jeffrey Harris, Scott E. Hudson, and Jennifer Mankoff. 2016. Helping hands: Requirements for a prototyping methodology for upper-limb prosthetics users. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 1769--1780.

Digital Library

[25]

Chandrika Jayant, Hanjie Ji, Samuel White, and Jeffrey P. Bigham. 2011. Supporting blind photography. In The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '11). ACM, NY, NY, USA, 203--210.

Digital Library

[26]

Shaun K. Kane and Jeffrey P. Bigham. 2014. Tracking @Stemxcomet: Teaching programming to blind students via 3D printing, crisis management, and twitter. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14). ACM, NY, NY, USA, 247--252.

Digital Library

[27]

Jeeeun Kim and Tom Yeh. 2015. Toward 3D-printed movable tactile pictures for children with visual impairments. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 2815--2824.

Digital Library

[28]

Benjamin Lafreniere, Tovi Grossman, Fraser Anderson, Justin Matejka, Heather Kerrick, Danil Nagy, Lauren Vasey, Evan Atherton, Nicholas Beirne, Marcelo H. Coelho, Nicholas Cote, Steven Li, Andy Nogueira, Long Nguyen, Tobias Schwinn, James Stoddart, David Thomasson, Ray Wang, Thomas White, David Benjamin, Maurice Conti, Achim Menges, and George Fitzmaurice. 2016. Crowdsourced fabrication. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 15--28.

Digital Library

[29]

Walter S. Lasecki, Phyo Thiha, Yu Zhong, Erin Brady, and Jeffrey P. Bigham. 2013. Answering visual questions with conversational crowd assistants. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). ACM, NY, NY, USA, Article 18, 8 pages.

Digital Library

[30]

Roberto Manduchi and James M. Coughlan. 2014. The last meter: Blind visual guidance to a target. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 3113--3122.

Digital Library

[31]

Samantha McDonald, Niara Comrie, Erin Buehler, Nicholas Carter, Braxton Dubin, Karen Gordes, Sandy McCombe-Waller, and Amy Hurst. 2016. Uncovering challenges and opportunities for 3D printing assistive technology with physical therapists. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 131--139.

Digital Library

[32]

Samantha McDonald, Joshua Dutterer, Ali Abdolrahmani, Shaun K. Kane, and Amy Hurst. 2014. Tactile aids for visually impaired graphical design education. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '14). ACM, NY, NY, USA, 275--276.

Digital Library

[33]

George A. Miller. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review 63, 2 (1956), 81.

[34]

Marius Muja and David G. Lowe. 2009. Fast approximate nearest neighbors with automatic algorithm configuration. In International Conference on Computer Vision Theory and Application VISSAPP'09). INSTICC Press, 331--340.

[35]

Jeffrey Nichols, Brad A. Myers, Michael Higgins, Joseph Hughes, Thomas K. Harris, Roni Rosenfeld, and Mathilde Pignol. 2002. Generating remote control interfaces for complex appliances. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (UIST '02). ACM, NY, NY, USA, 161--170.

Digital Library

[36]

Raf Ramakers, Fraser Anderson, Tovi Grossman, and George Fitzmaurice. 2016. RetroFab: A design tool for retrofitting physical interfaces using actuators, sensors and 3D printing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 409--419.

Digital Library

[37]

Lei Shi, Idan Zelzer, Catherine Feng, and Shiri Azenkot. 2016. Tickers and talker: An accessible labeling toolkit for 3D printed models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 4896--4907.

Digital Library

[38]

Saiganesh Swaminathan, Thijs Roumen, Robert Kovacs, David Stangl, Stefanie Mueller, and Patrick Baudisch. 2016. Linespace: A sensemaking platform for the blind. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 2175--2185.

Digital Library

[39]

Brandon Taylor, Anind Dey, Dan Siewiorek, and Asim Smailagic. 2016. Customizable 3D printed tactile maps as interactive overlays. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 71--79.

Digital Library

[40]

United States Access Board 2016. Advancing full access and inclusion for all. https://www.access-board.gov/guidelines-and-standards. (2016).

[41]

Gregg Vanderheiden and Jutta Treviranus. 2011. Creating a global public inclusive infrastructure. In International Conference on Universal Access in Human-Computer Interaction. Springer, 517--526.

Digital Library

[42]

Marynel Vázquez and Aaron Steinfeld. 2014. An assisted photography framework to help visually impaired users properly aim a camera. ACM Transactions on Computer-Human Interaction 21, 5, Article 25 (Nov. 2014), 29 pages.

Digital Library

[43]

Samuel White, Hanjie Ji, and Jeffrey P. Bigham. 2010. EasySnap: Real-time audio feedback for blind photography. In Adjunct Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, NY, NY, USA, 409--410.

Digital Library

[44]

Wikipedia. 2016. Homography (computer vision) - Wikipedia, The Free Encyclopedia. (2016). https://en.wikipedia.org/w/index.php?title=Homography_(computer_vision)&oldid=752434320 {Online; accessed 1-December-2016}.

[45]

Yu Zhong, Pierre J. Garrigues, and Jeffrey P. Bigham. 2013. Real time object scanning using a mobile phone and cloud-based visual search engine. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). ACM, NY, NY, USA, Article 20, 8 pages.

Digital Library

[46]

Yu Zhong, Walter S. Lasecki, Erin Brady, and Jeffrey P. Bigham. 2015. RegionSpeak: Quick comprehensive spatial descriptions of complex images for blind users. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 2353--2362.

Digital Library

Cited By

Li FWang ACarrington PKane S(2024)A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking InstructionsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675662(1-15)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675662
Campos Zamora DHe LFroehlich J(2024)MobiPrint: A Mobile 3D Printer for Environment-Scale Design and FabricationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676459(1-10)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676459
Cho HSendhilnathan NNebeling MWang TPadmanabhan PBrowder JLindlbauer DJonker TTodi K(2024)SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676384(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676384
Show More Cited By

Index Terms

Facade: Auto-generating Tactile Interfaces to Appliances
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Social and professional topics
  1. Professional topics
    1. Computing profession
      1. Assistive technologies
  2. User characteristics
    1. People with disabilities

Recommendations

Making Real-World Interfaces Accessible Through Crowdsourcing, Computer Vision, and Fabrication
W4A '17: Proceedings of the 14th International Web for All Conference

The world is full of physical interfaces that are inaccessible to blind people, from microwaves and information kiosks to thermostats and checkout terminals. Blind people cannot independently use such devices without at least first learning their layout,...
Crowd-AI Systems for Non-Visual Information Access in the Real World
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

The world is full of information, interfaces and environments that are inaccessible to blind people. When navigating indoors, blind people are often unaware of key visual information, such as posters, signs, and exit doors. When accessing specific ...
Crowd-AI Systems for Non-Visual Information Access in the Real World
UIST '18 Adjunct: Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

The world is full of information, interfaces and environments that are inaccessible to blind people. When navigating indoors, blind people are often unaware of key visual information, such as posters, signs, and exit doors. When accessing specific ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

May 2017

7138 pages

ISBN:9781450346559

DOI:10.1145/3025453

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation
National Institute on Disability, Independent Living, and Rehabilitation Research
Google

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
1,152
Total Downloads

Downloads (Last 12 months)162
Downloads (Last 6 weeks)16

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li FWang ACarrington PKane S(2024)A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking InstructionsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675662(1-15)Online publication date: 27-Oct-2024
Campos Zamora DHe LFroehlich J(2024)MobiPrint: A Mobile 3D Printer for Environment-Scale Design and FabricationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676459(1-10)Online publication date: 13-Oct-2024
Cho HSendhilnathan NNebeling MWang TPadmanabhan PBrowder JLindlbauer DJonker TTodi K(2024)SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676384(1-19)Online publication date: 13-Oct-2024
Mei YJones BCascaval DMankoff JVouga ESchulz A(2024)FabHacks: Transform Everyday Objects into Home Hacks Leveraging a Solver-aided DSLProceedings of the 9th ACM Symposium on Computational Fabrication10.1145/3639473.3665788(1-16)Online publication date: 7-Jul-2024
Kwon NLu QQazi MLiu JOh CKong SKim J(2024)AccessLens: Auto-detecting Inaccessibility of Everyday ObjectsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642767(1-17)Online publication date: 11-May-2024
Li FLiu MKane SCarrington P(2024)A Contextual Inquiry of People with Vision Impairments in CookingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642233(1-14)Online publication date: 11-May-2024
Bleau MJaiswal AHolzhey PWittich W(2024)3D printing as assistive technology for individuals with deafblindness: perspectives of rehabilitation professionalsDisability and Rehabilitation: Assistive Technology10.1080/17483107.2024.2431630(1-14)Online publication date: 21-Nov-2024
Myers B(2024)Pick, Click, Flick!undefinedOnline publication date: 14-Mar-2024
Regal GMaidhof CPuthenkalam JKhermayer ŽPavšek KSackl A(2023)Feel the Art: Digital, Tangible Postcards for Accessible Cultural ExperiencesProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3631877(559-561)Online publication date: 3-Dec-2023
Mankoff JMack KWiese JCrawford KHamidi F(2023)A11yFutures: Envisioning the Future of Accessibility ResearchProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3615652(1-4)Online publication date: 22-Oct-2023
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents