Find My Things: Personalized Accessibility through Teachable AI for People who are Blind or Low Vision
Authors: 
Linda Yilin Wen, Cecily Morrison, Martin Grayson, Rita Faia Marques, Daniela Massiceti, Camilla Longden, 
Edward CutrellAuthors Info & Claims
Linda Yilin Wen, Cecily Morrison, Martin Grayson, Rita Faia Marques, Daniela Massiceti, Camilla Longden, Edward CutrellAuthors Info & ClaimsArticle No.: 403, Pages 1 - 6
Abstract
The opportunity for artificial intelligence, or AI, to enable accessibility is rapidly growing, but widely impactful applications can be challenging to build given the diversity of user need within and across disability communities. Teachable AI systems give users with disabilities a way to leverage the power of AI to personalize applications for their own specific needs. We demonstrate Find My Things as an end-to-end example of applying Teachable AI systems to address the diversity of accessibility needs. An application that can be taught by people who are blind or low vision to find their personal things, Find My Things illustrates the potential Teachable AI holds for accessibility.
Supplemental Material
MP4 File - Demonstration Video
Demonstration Video
- Download
- 196.81 MB
- Transcript
References
[1]
Dragan Ahmetovic, Daisuke Sato, Uran Oh, Tatsuya Ishihara, Kris Kitani, and Chieko Asakawa. 2020. Recog: Supporting blind people in recognizing personal objects. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[2]
Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI magazine 35, 4: 105–120.
[3]
C. Andrews. 2014. Accessible Participatory Design: Engaging and Including Visually Impaired Participants. In Inclusive Designing. Springer International Publishing.
[4]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 2017 ICML International Conference on Machine Learning, 1126-1135.
[5]
Jonggi Hong, Jaina Gandhi, Ernest Essuah Mensah, Farnaz Zamiri Zeraati, Ebrima Haddy Jarjue, Kyungjun Lee, and Hernisa Kacorri. 2022. Blind users accessing their training images in teachable object recognizers. ASSETS. ACM Conference on Assistive Technologies 2022. https://doi.org/10.1145/3517428.3544824
[6]
Jonggi Hong, Kyungjun Lee, June Xu, and Hernisa Kacorri. 2020. Crowdsourcing the Perception of Machine Teaching. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
[7]
Hernisa Kacorri. 2017. Teachable machines for accessibility. ACM SIGACCESS accessibility and computing, 119: 10–18.
[8]
Hernisa Kacorri, Kris M. Kitani, Jeffrey P. Bigham, and Chieko Asakawa. 2017. People with visual impairment training personal object recognizers: Feasibility and challenges. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 5839–5849.
[9]
Simon Katan, Mick Grierson, and Rebecca Fiebrink. 2015. Using interactive machine learning to support interface development through workshops with disabled people. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2702123.2702474
[10]
Kyungjun Lee, Abhinav Shrivastava, and Hernisa Kacorri. 2020. Hand-priming in object localization for assistive egocentric vision. IEEE Winter Conference on Applications of Computer Vision. IEEE Winter Conference on Applications of Computer Vision 2020: 3411–3421.
[11]
Daniela Massiceti, Luisa Zintgraf, John Bronskill, Lida Theodorou, Matthew Tobias Harris, Edward Cutrell, Cecily Morrison, Katja Hofmann, and Simone Stumpf. 2021. ORBIT: A real-world few-shot dataset for teachable object recognition. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv48922.2021.01064
[12]
Yuri Nakao and Yusuke Sugano. 2020. Use of Machine Learning by Non-Expert DHH People: Technological Understanding and Sound Perception. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society. 1–12.
[13]
Gonzalo Ramos, Christopher Meek, Patrice Simard, Jina Suh, and Soroush Ghorashi. 2020. Interactive machine teaching: a human-centered approach to building machine-learned models. Human-computer interaction 35, 5–6: 413–451.
[14]
Téo Sanchez, Baptiste Caramiaux, Jules Françoise, Frédéric Bevilacqua, and Wendy E. Mackay. 2021. How do People Train a Machine? Proceedings of the ACM on human-computer interaction 5, CSCW1: 1–26.
[15]
Dan Shapiro. 2005. Participatory design: the will to succeed. In Proceedings of the CC Conference on Critical Computing. 29–38.
[16]
Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical Networks for Few-shot Learning. In Advances in Neural Information Processing Systems 30, 1 - 11.
[17]
M. Steven, Ping Goodman, Dhruv Liu, Emma J. Jain, Jon E. Mcdonnell, and Leah Froehlich. 2021. Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5: 1–23.
[18]
Mingxing Tan and Quoc V. Le. 2019. EfficientNet: Rethinking model scaling for convolutional Neural Networks. arXiv [cs.LG]. Retrieved from http://arxiv.org/abs/1905.11946
[19]
Luisa M. Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. 2018. Fast Context Adaptation via Meta-Learning. In Proceedings of the ICML Conference on Machine Learning, 7693-7702.
Index Terms
- Find My Things: Personalized Accessibility through Teachable AI for People who are Blind or Low Vision
Recommendations
Understanding Personalized Accessibility through Teachable AI: Designing and Evaluating Find My Things for People who are Blind or Low Vision
ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and AccessibilityThe opportunity for artificial intelligence, or AI, to enable accessibility is rapidly growing, but widely impactful applications can be challenging to build given the diversity of user need within and across disability communities. Teachable AI systems ...
Exploring the Opportunities and Challenges with Exercise Technologies for People who are Blind or Low-Vision
ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & AccessibilityPeople who are blind or low-vision may have a harder time participating in exercise due to inaccessibility or lack of experience. We employed Value Sensitive Design (VSD) to explore the potential of technology to enhance exercise for people who are ...
Comments
Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Qualifiers
- Demonstration
- Research
- Refereed limited
Data Availability
Demonstration Video: Demonstration Videohttps://dl.acm.org/doi/10.1145/3613905.3648641#3613905.3648641-demo-video.mp4
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 243Total Downloads
- Downloads (Last 12 months)243
- Downloads (Last 6 weeks)54
Reflects downloads up to 10 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format