research-article

Pedestrian Detection with Wearable Cameras for the Blind: A Two-way Perspective

Authors:

Hernisa Kacorri,

Chieko AsakawaAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 12

https://doi.org/10.1145/3313831.3376398

Published: 23 April 2020 Publication History

Abstract

Blind people have limited access to information about their surroundings, which is important for ensuring one's safety, managing social interactions, and identifying approaching pedestrians. With advances in computer vision, wearable cameras can provide equitable access to such information. However, the always-on nature of these assistive technologies poses privacy concerns for parties that may get recorded. We explore this tension from both perspectives, those of sighted passersby and blind users, taking into account camera visibility, in-person versus remote experience, and extracted visual information. We conduct two studies: an online survey with MTurkers (N=206) and an in-person experience study between pairs of blind (N=10) and sighted (N=40) participants, where blind participants wear a working prototype for pedestrian detection and pass by sighted participants. Our results suggest that both of the perspectives of users and bystanders and the several factors mentioned above need to be carefully considered to mitigate potential social tensions.

Supplementary Material

MP4 File (paper271vf.mp4)

Supplemental video

Download
52.77 MB

MP4 File (paper271pv.mp4)

Preview video

Download
13.91 MB

MP4 File (a271-lee-presentation.mp4)

Download
77.75 MB

References

[1]

OrCam MyEye 2. 2019. Help people who are blind or partially sighted. (2019). https://www.orcam.com/en/

[2]

David Ahlström, Khalad Hasan, and Pourang Irani. 2014. Are you comfortable doing that?: acceptance studies of around-device gestures in and for public settings. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services. ACM, 193--202.

Digital Library

[3]

Tousif Ahmed, Apu Kapadia, Venkatesh Potluri, and Manohar Swaminathan. 2018. Up to a limit?: Privacy concerns of bystanders and their willingness to share additional information with visually impaired users of assistive technologies. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 89.

Digital Library

[4]

Aira. 2019. Connecting you to real people instantly to simplify daily life. (2019). https://aira.io/

[5]

Shurug Al-Khalifa and Muna Al-Razgan. 2016. Ebsar: Indoor guidance for the visually impaired. Computers & Electrical Engineering 54 (2016), 26--39.

Digital Library

[6]

Vuzix Blade. 2019. Augment Reality (AR) Smart Glasses for the Consumer. (2019). https://www.vuzix.com/products/blade-smart-glasses

[7]

Stacy M Branham, Ali Abdolrahmani, William Easley, Morgan Scheuerman, Erick Ronquillo, and Amy Hurst. 2017. Is Someone There? Do They Have a Gun: How Visual Information about Others Can Improve Personal Safety Management for Blind Individuals. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 260--269.

Digital Library

[8]

Jiankang Deng, Jia Guo, Xue Niannan, and Stefanos Zafeiriou. 2019. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. In CVPR.

[9]

Tamara Denning, Zakariya Dehlawi, and Tadayoshi Kohno. 2014. In situ with bystanders of augmented reality glasses: Perspectives on recording and privacy-mediating technologies. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 2377--2386.

Digital Library

[10]

Mat D Duerden and Peter A Witt. 2010. The impact of direct and indirect experiences on the development of environmental knowledge, attitudes, and behavior. Journal of environmental psychology 30, 4 (2010), 379--392.

[11]

eSight. 2019. Electronic glasses for the legally blind. (2019). https://esighteyewear.com/

[12]

Alexander Fiannaca, Ilias Apostolopoulous, and Eelke Folmer. 2014. Headlock: a wearable navigation aid that helps blind cane users traverse large open spaces. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility. ACM, 19--26.

Digital Library

[13]

Leah Findlater, Steven Goodman, Yuhang Zhao, Shiri Azenkot, and Margot Hanley. 2019. Fairness Issues in AI Systems that Augment Sensory Abilities. arXiv preprint arXiv:1908.07333 (2019).

[14]

Steven Goodman, Dhruv Jain, Jon Froehlich, Brock Craft, and Leah Findlater. 2019. Social Tensions with Head-Mounted Displays for Accessibility. (2019).

[15]

Nicolas Gourier, Daniela Hall, and James L Crowley. 2004. Estimating face orientation from robust detection of salient facial features. In ICPR International Workshop on Visual Observation of Deictic Gestures. Citeseer.

[16]

Chien-Ju Ho, Aleksandrs Slivkins, Siddharth Suri, and Jennifer Wortman Vaughan. 2015. Incentivizing high quality crowdwork. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 419--429.

Digital Library

[17]

Chandrika Jayant, Hanjie Ji, Samuel White, and Jeffrey P Bigham. 2011. Supporting blind photography. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility. ACM, 203--210.

Digital Library

[18]

Yongsik Jin, Jonghong Kim, Bumhwi Kim, Rammohan Mallipeddi, and Minho Lee. 2015. Smart cane: Face recognition system for blind. In Proceedings of the 3rd International Conference on Human-Agent Interaction. ACM, 145--148.

Digital Library

[19]

Norene Kelly. 2016. The WEAR Scale: Development of a measure of the social acceptability of a wearable device. (2016).

[20]

Mohammad Kianpisheh, Franklin Mingzhe Li, and Khai N. Truong. 2019. Face Recognition Assistant for People with Visual Impairments. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 3, Article Article 90 (Sept. 2019), 24 pages.

Digital Library

[21]

Priyan Malarvizhi Kumar, Ushadevi Gandhi, R Varatharajan, Gunasekaran Manogaran, R Jidhesh, and Thanjai Vadivel. 2017. Intelligent face recognition and navigation system using neural learning for smart security in Internet of Things. Cluster Computing (2017), 1--12.

[22]

Meethu Malu and Leah Findlater. 2015. Personalized, wearable control of a head-mounted display for users with upper body motor impairments. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 221--230.

Digital Library

[23]

Gerard McAtamney and Caroline Parker. 2006. An examination of the effects of a wearable display on informal face-to-face communication. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 45--54.

Digital Library

[24]

Róisín McNaney, John Vines, Daniel Roggen, Madeline Balaam, Pengfei Zhang, Ivan Poliakov, and Patrick Olivier. 2014. Exploring the acceptability of google glass as an everyday assistive device for people with parkinson's. In Proceedings of the sigchi conference on human factors in computing systems. ACM, 2551--2554.

Digital Library

[25]

Calkin S Montero, Jason Alexander, Mark T Marshall, and Sriram Subramanian. 2010. Would you do that?: understanding social acceptance of gestural interfaces. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. ACM, 275--278.

Digital Library

[26]

Halley Profita, Reem Albaghli, Leah Findlater, Paul Jaeger, and Shaun K Kane. 2016. The AT effect: how disability affects the perceived social acceptability of head-mounted display use. In proceedings of the 2016 CHI conference on human factors in computing systems. ACM, 4884--4895.

Digital Library

[27]

Halley P. Profita, James Clawson, Scott Gilliland, Clint Zeagler, Thad Starner, Jim Budd, and Ellen Yi-Luen Do. 2013. Don't mind me touching my wrist: a case study of interacting with on-body technology in public. In Proceedings of the 2013 International Symposium on Wearable Computers. ACM, 89--96.

Digital Library

[28]

Philipp A Rauschnabel and Young K Ro. 2016. Augmented reality smart glasses: An investigation of technology acceptance drivers. International Journal of Technology Marketing 11, 2 (2016), 123--148.

[29]

Julie Rico and Stephen Brewster. 2009. Gestures all around us: user differences in social acceptability perceptions of gesture based interfaces. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 64.

Digital Library

[30]

Julie Rico and Stephen Brewster. 2010. Usable gestures for mobile interfaces: evaluating social acceptability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 887--896.

Digital Library

[31]

Larry D Rosen, Kelly Whaling, L Mark Carrier, Nancy A Cheever, and Jeffrey Rokkum. 2013. The media and technology usage and attitudes scale: An empirical investigation. Computers in human behavior 29, 6 (2013), 2501--2511.

[32]

M Saquib Sarfraz, Angela Constantinescu, Melanie Zuzej, and Rainer Stiefelhagen. 2017. A multimodal assistive system for helping visually impaired in social interactions. Informatik-Spektrum 40, 6 (2017), 540--545.

[33]

Valentin Schwind, Jens Reinhardt, Rufat Rzayev, Niels Henze, and Katrin Wolf. 2018. On the Need for Standardized Methods to Study the Social Acceptability of Emerging Technologies. In CHI'18 Workshop on (Un) Acceptable!?!-Re-thinking the Social Acceptability of Emerging Technologies, Vol. 6.

[34]

SeeingAI. 2019. Talking camera app for those with a visual impairment. (2019). https://www.microsoft.com/en-us/seeing-ai

[35]

Lee Stearns, Leah Findlater, and Jon E Froehlich. 2018. Design of an Augmented Reality Magnification Aid for Low Vision Users. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 28--39.

Digital Library

[36]

Lee Stearns and Anja Thieme. 2018. Automated Person Detection in Dynamic Scenes to Assist People with Vision Impairments: An Initial Investigation. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 391--394.

Digital Library

[37]

Katrin Wolf, Albrecht Schmidt, Agon Bexheti, and Marc Langheinrich. 2014. Lifelogging: You're wearing a camera? IEEE Pervasive Computing 13, 3 (2014), 8--12.

[38]

Qingguo Xu, Sen-ching Samson Cheung, and Neelkamal Soares. 2015a. LittleHelper: an augmented reality glass application to assist individuals with autism in job interview. In 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE, 1276--1279.

[39]

Qianli Xu, Michal Mukawa, Liyuan Li, Joo Hwee Lim, Cheston Tan, Shue Ching Chia, Tian Gan, and Bappaditya Mandal. 2015b. Exploring users' attitudes towards social interaction assistance on Google Glass. In Proceedings of the 6th Augmented Human international conference. ACM, 9--12.

Digital Library

[40]

Chuang-Wen You, Ya-Fang Lin, Elle Luo, Hung-Yeh Lin, and Hsin-Liu Cindy Kao. 2019. Understanding social perceptions towards interacting with on-skin interfaces in public. In Proceedings of the 23rd International Symposium on Wearable Computers. ACM, 244--253.

Digital Library

[41]

Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 10 (2016), 1499--1503.

[42]

Yuhang Zhao, Sarit Szpiro, Lei Shi, and Shiri Azenkot. 2019. Designing and Evaluating a Customizable Head-Mounted Vision Enhancement System for People with Low Vision. ACM Trans. Access. Comput. 12, 4, Article Article 15 (Dec. 2019), 46 pages.

Digital Library

[43]

Yuhang Zhao, Shaomei Wu, Lindsay Reynolds, and Shiri Azenkot. 2018. A Face Recognition Application for People with Visual Impairments: Understanding Use Beyond the Lab. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 215.

Digital Library

[44]

Annuska Zolyomi, Anushree Shukla, and Jaime Snyder. 2017. Technology-Mediated Sight: A Case Study of Early Adopters of a Low Vision Assistive Technology. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 220--229.

Digital Library

Cited By

Theodorou PTsiligkos KMeliones A(2024)Challenges in Acceptance of Smartphone-Based Assistive Technologies: Extending the UTAUT Model for People With Visual ImpairmentsJournal of Visual Impairment & Blindness10.1177/0145482X241231990118:1(18-30)Online publication date: 6-Mar-2024
https://doi.org/10.1177/0145482X241231990
Panwar MDhankhar ARajoria HSoreng JBatsyas RKharangarh P(2024)Review—Innovations in Flexible Sensory Devices for the Visually ImpairedECS Journal of Solid State Science and Technology10.1149/2162-8777/ad658813:7(077011)Online publication date: 30-Jul-2024
https://doi.org/10.1149/2162-8777/ad6588
Stampf ASasalovici MMeinhardt LColley MGiss MRukzio E(2024)Move, Connect, Interact: Introducing a Design Space for Cross-Traffic InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785808:3(1-40)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678580
Show More Cited By

Index Terms

Pedestrian Detection with Wearable Cameras for the Blind: A Two-way Perspective
1. Human-centered computing
  1. Accessibility
    1. Accessibility technologies
    2. Empirical studies in accessibility

Recommendations

Accessing Passersby Proxemic Signals through a Head-Worn Camera: Opportunities and Limitations for the Blind
ASSETS '21: Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility

The spatial behavior of passersby can be critical to blind individuals to initiate interactions, preserve personal space, or practice social distancing during a pandemic. Among other use cases, wearable cameras employing computer vision can be used to ...
Visual challenges in the everyday lives of blind people
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

The challenges faced by blind people in their everyday lives are not well understood. In this paper, we report on the findings of a large-scale study of the visual questions that blind people would like to have answered. As part of this year-long study, ...
Regulating Personal Cameras for Disabled People and People with Deafblindness: Implications for HCI and Accessible Computing
ASSETS '21: Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility

In this experience paper we consider the relevance of social policy to the design of personal cameras for accessibility. As researchers with different backgrounds, in disability studies, accessible computing, AI-based systems, HCI and disability policy, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Institute on Disability, Independent Living, and Rehabilitation Research

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
899
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Theodorou PTsiligkos KMeliones A(2024)Challenges in Acceptance of Smartphone-Based Assistive Technologies: Extending the UTAUT Model for People With Visual ImpairmentsJournal of Visual Impairment & Blindness10.1177/0145482X241231990118:1(18-30)Online publication date: 6-Mar-2024
https://doi.org/10.1177/0145482X241231990
Panwar MDhankhar ARajoria HSoreng JBatsyas RKharangarh P(2024)Review—Innovations in Flexible Sensory Devices for the Visually ImpairedECS Journal of Solid State Science and Technology10.1149/2162-8777/ad658813:7(077011)Online publication date: 30-Jul-2024
https://doi.org/10.1149/2162-8777/ad6588
Stampf ASasalovici MMeinhardt LColley MGiss MRukzio E(2024)Move, Connect, Interact: Introducing a Design Space for Cross-Traffic InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785808:3(1-40)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678580
Hong JKacorri H(2024)Understanding How Blind Users Handle Object Recognition Errors: Strategies and ChallengesProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675635(1-15)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675635
Kamikubo RZamiri Zeraati FLee KKacorri H(2024)AccessShare: Co-designing Data Access and Sharing with Blind PeopleProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675612(1-16)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675612
Teng YCourtien CRios DTseng YGibson JAziz MReyna AVaish RSmith B(2024)Help Supporters: Exploring the Design Space of Assistive Technologies to Support Face-to-Face Help Between Blind and Sighted StrangersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642816(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642816
Cai SRam AGou ZShaikh MChen YWan YHara KZhao SHsu D(2024)Navigating Real-World Challenges: A Quadruped Robot Guiding System for Visually Impaired People in Diverse EnvironmentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642227(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642227
Theodorou PTsiligkos KMeliones A(2023)Multi-Sensor Data Fusion Solutions for Blind and Visually Impaired: Research and Commercial Navigation Applications for Indoor and Outdoor SpacesSensors10.3390/s2312541123:12(5411)Online publication date: 7-Jun-2023
https://doi.org/10.3390/s23125411
O'Hagan JSaeghe PGugenheimer JMedeiros DMarky KKhamis MMcGill M(2023)Privacy-Enhancing Technology and Everyday Augmented RealityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35695016:4(1-35)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569501
Kamikubo RLee KKacorri H(2023)Contributing to Accessibility Datasets: Reflections on Sharing Study Data by Blind PeopleProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581337(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581337
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents