research-article

BSpeak: An Accessible Voice-based Crowdsourcing Marketplace for Low-Income Blind People

Authors:

Aditya Vashistha,

Richard AndersonAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 57, Pages 1 - 13

https://doi.org/10.1145/3173574.3173631

Published: 19 April 2018 Publication History

Abstract

BSpeak is an accessible crowdsourcing marketplace that enables blind people in developing regions to earn money by transcribing audio files through speech. We examine accessibility and usability barriers that 15 first-time users, who are low-income and blind, experienced while completing transcription tasks on BSpeak and Mechanical Turk (MTurk). Our mixed-methods analysis revealed severe accessibility barriers in MTurk due to the absence of landmarks, unlabeled UI elements, and improper use of HTML headings. Compared to MTurk, participants found BSpeak significantly more accessible and usable, and completed tasks with higher accuracy in lesser time due to its voice-based implementation. In a two-week field deployment of BSpeak in India, 24 low-income blind users earned rupee 7,310 by completing over 16,000 transcription tasks to yield transcriptions with 87% accuracy. Through our analysis of BSpeak's strengths and weaknesses, we provide recommendations for designing crowdsourcing marketplaces for low-income blind people in resource-constrained settings.

Supplementary Material

suppl.mov (pn1370-file5.mp4)

Supplemental video

Download
1.94 MB

MP4 File (pn1370.mp4)

Download
207.03 MB

References

[1]

2009. Employment of Disabled People in India. Technical Report. Diversity and Equal Opportunity Centre (DEOC).

[2]

2014. WHO | Visual impairment and blindness. (2014). http: //www.who.int/mediacentre/factsheets/fs282/en/.

[3]

2016. Get Paid to Click: 10 Legit Sites That Pay You to Click on Ads | SurveySatrap. (Feb. 2016). http://surveysatrap.com/get-paid-to-click/.

[4]

2016. National Institute for the Visually Handicapped Annual Report 2015--16. Technical Report. Department OF Empowerment of Persons with Disabilities, Ministry of Social Justic and Empowerment, Government of India. http://nivh.gov.in/index.php/annual-report

[5]

2016. Quick Transcription Service. (2016). http://www.quicktranscriptionservice.com/ Hindi-Transcription.html.

[6]

2016. Scripts Complete. (2016). http://scriptscomplete.com/ Hindi-Transcription-Services.php.

[7]

2016. Transcription Services Us. (2016). http://www.transcription-services-us.com/ Language-Transcription-Rates.php.

[8]

2017. Accessibility | Android Developers. (2017). https://developer.android.com/guide/topics/ui/ accessibility/index.html.

[9]

2017. Amazon Mechanical Turk. (2017). https://www.mturk.com/mturk/welcome.

[10]

2017. AWS Developer Forums: MTurk CENSUS: About how many workers were on Mechanical Turk in 2010? (2017). https://forums.aws.amazon.com/thread. jspa?threadID=58891.

[11]

2017. Be My Eyes Lend Your Eyes to the Blind. (2017). http://bemyeyes.com.

[12]

2017. CLOUD SPEECH API: Speech to text conversion powered by machine learning. (2017). https://cloud.google.com/speech/.

[13]

2017. CrowdFlower: AI for your business. (2017). https://www.crowdflower.com/.

[14]

2017a. Four Dollar Click | Earn Money by Viewing Ads. (2017). https://fourdollarclick.com/.

[15]

2017. H59: Using the link element and navigation tools | Techniques for WCAG 2.0. (2017). https://www.w3.org/TR/WCAG20-TECHS/H59.html

[16]

2017. India Average Daily Wage Rate Forecast 2016--2020. (2017). http://www.tradingeconomics. com/india/wages/forecast.

[17]

2017. Jana. (2017). https://www.jana.com/.

[18]

2017. Labeling Regions Page Structure WAI Web Accessibility Tutorials. (2017). https://www.w3.org/ WAI/tutorials/page-structure/labels/.

[19]

2017. Liv.ai. (2017). https://liv.ai/.

[20]

2017. mSurvey. (2017). https://www.msurvey.co/.

[21]

2017. Paytm. (2017). https://paytm.com/.

[22]

2017b. Poverty & Equity Data | India | The World Bank. (2017). http: //povertydata.worldbank.org/poverty/country/IND, urldate = 2016-09--30.

[23]

2017. Samasource. (2017). http://www.samasource.org/.

[24]

2017c. Two Dollar Click : Get Paid To Click. (2017). http: //www.twodollarclick.com/index.php?view=click.

[25]

2017. Using ARIA landmarks to identify regions of a page - WCAG WG. (2017). https://www.w3.org/WAI/GL/wiki/Using_ARIA_landmarks_to_identify_regions_of_a_page

[26]

2017. WCAG Overview | Web Accessibility Initiative (WAI) | W3C. (2017). https://www.w3.org/WAI/intro/wcag.

[27]

2018. Babajob. (2018). http://www.babajob.com/.

[28]

2018. CGNet Swara. (2018). http://cgnetswara.org/.

[29]

2018. Gram Vaani. (2018). http://www.gramvaani.org/.

[30]

2018a. Hotlines & Helplines | National AIDS Control Organization | MoHFW | GoI. (2018). http://naco.gov.in/hotlines-helplines.

[31]

2018b. Kisan Call center. (2018). http://dackkms.gov.in.

[32]

2018c. Minimum Wages | Chief Labour Commissioner. (2018). http://clc.gov.in/clc/min-wages.

[33]

2018d. NCW::Helpline. (2018). http://ncw.nic.in/frmhelpline.aspx.

[34]

Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C. Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, and Tom Yeh. 2010. VizWiz: Nearly Real-time Answers to Visual Questions. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, New York, NY, USA, 333--342.

Digital Library

[35]

Erin Brady, Meredith Ringel Morris, and Jeffrey P. Bigham. 2015. Gauging Receptiveness to Social Microvolunteering. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1055--1064.

Digital Library

[36]

Erin Brady, Meredith Ringel Morris, Yu Zhong, Samuel White, and Jeffrey P. Bigham. 2013a. Visual Challenges in the Everyday Lives of Blind People. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2117--2126.

Digital Library

[37]

Erin L. Brady, Yu Zhong, Meredith Ringel Morris, and Jeffrey P. Bigham. 2013b. Investigating the Appropriateness of Social Network Question Asking As a Resource for Blind Users. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (CSCW '13). ACM, New York, NY, USA, 1225--1236.

Digital Library

[38]

V. Braun and V. Clarke. 2006. Using Thematic Analysis in Psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101.

[39]

Stacy Butler, Adele Crudden, and William Sansing. 2005. Overcoming barriers to employment: Strategies of rehabilitation providers. Journal of Visual Impairment & Blindness (JVIB) 99, 06 (2005).

[40]

Anthony Candela and Karen Wolffe. 2002. A qualitative analysis of employers' experiences with visually impaired workers. Journal of Visual Impairment & Blindness (JVIB) 96, 09 (2002).

[41]

Adele Crudden, Lynn W McBroom, Amy L Skinner, and J Elton Moore. 1998. Comprehensive Examination of Barriers to Employment among Persons Who Are Blind or Visually Impaired. (1998).

[42]

Nathan Eagle. 2009. Txteagle: Mobile Crowdsourcing. In Proceedings of HCI International. Springer-Verlag.

Digital Library

[43]

Kristin Skeide Fuglerud, Ingvar Tjøstheim, Birkir Růnar Gunnarsson, and Morten Tollefsen. 2012. Use of Social Media by People with Visual Impairments: Usage Levels, Attitudes and Barriers. In Proceedings of the 13th International Conference on Computers Helping People with Special Needs - Volume Part I (ICCHP'12). Springer-Verlag, Berlin, Heidelberg, 565--572.

Digital Library

[44]

Yashesh Gaur, Florian Metze, and Jeffrey P. Bigham. 2016. Manipulating Word Lattices to Incorporate Human Corrections. In INTERSPEECH.

[45]

Yvonne HH Goertz, Brigitte AG van Lierop, Inge Houkes, and Frans JN Nijhuis. 2010. Factors related to the employment of visually impaired persons: A systematic literature review. Journal of Visual Impairment & Blindness 104, 7 (2010), 404.

[46]

Aakar Gupta, William Thies, Edward Cutrell, and Ravin Balakrishnan. 2012. mClerk: Enabling Mobile Crowdsourcing in Developing Regions. In Proceedings of CHI.

Digital Library

[47]

Rohit Khanna, Usha Raman, and Gullapalli N. Rao. 2007. Blindness and poverty in India: the way forward. Clinical & Experimental Optometry 90, 6 (Nov. 2007), 406--414.

[48]

Shashank Khanna, Aishwarya Ratan, James Davis, and William Thies. 2010. Evaluating and Improving the Usability of Mechanical Turk for Low-income Workers in India. In Proceedings of the First ACM Symposium on Computing for Development (ACM DEV '10). ACM, New York, NY, USA, 12:1--12:10.

Digital Library

[49]

Ramya Naraharisetti and Marcia C. Castro. 2016. Factors associated with persons with disability employment in India: a cross-sectional study. BMC Public Health 16 (Oct. 2016), 1063.

[50]

Prayag Narula, David Rolnitzky, and Bjoern Hartmann. 2011. MobileWorks: A Mobile Crowdsourcing Platform for Workers at the Bottom of the Pyramid. In In Proceedings of HCOMP.

Digital Library

[51]

Joyojeet Pal and Meera Lakshmanan. 2012. Assistive Technology and the Employment of People with Vision Impairments in India. In Proceedings of the Fifth International Conference on Information and Communication Technologies and Development (ICTD '12). ACM, New York, NY, USA, 307--317.

Digital Library

[52]

D. Pascolini and S. P. Mariotti. 2012. Global estimates of visual impairment: 2010. British Journal of Ophthalmology 96, 5 (May 2012), 614--618.

[53]

PD Rumrill and CM Scheff. 1997. Impact of the ADA on the employment and promotion of persons who are visually impaired. Occupational Health and Industrial Medicine 6, 37 (1997), 311.

[54]

Alexander Shaw, Deborah Gold, and Karen Wolffe. 2007. Employment-Related Experiences of Youths Who Are Visually Impaired: How Are These Youths Faring? Journal of Visual Impairment & Blindness v101 (Jan. 2007), 7--21.

[55]

Aditya Vashistha, Erin Brady, William Thies, and Edward Cutrell. 2014. Educational Content Creation and Sharing by Low-Income Visually Impaired People in India. In Proceedings of the Fifth ACM Symposium on Computing for Development (ACM DEV-5 '14). ACM, New York, NY, USA, 63--72.

Digital Library

[56]

Aditya Vashistha, Edward Cutrell, Nicola Dell, and Richard Anderson. 2015. Social Media Platforms for Low-Income Blind People in India. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '15). ACM, New York, NY, USA, 259--272.

Digital Library

[57]

Aditya Vashistha, Pooja Sethi, and Richard Anderson. 2017. Respeak: A Voice-based, Crowd-powered Speech Transcription System. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 1855--1866.

Digital Library

[58]

Brian Wentz and Jonathan Lazar. 2011. Are Separate Interfaces Inherently Unequal?: An Evaluation with Blind Users of the Usability of Two Interfaces for a Social Networking Platform. In Proceedings of the 2011 iConference (iConference '11). ACM, New York, NY, USA, 91--97.

Digital Library

[59]

Yu Zhong, Walter S. Lasecki, Erin Brady, and Jeffrey P. Bigham. 2015. RegionSpeak: Quick Comprehensive Spatial Descriptions of Complex Images for Blind Users. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2353--2362.

Digital Library

[60]

Kathryn Zyskowski, Meredith Ringel Morris, Jeffrey P. Bigham, Mary L. Gray, and Shaun K. Kane. 2015. Accessible Crowdwork?: Understanding the Value in and Challenge of Microtask Employment for People with Disabilities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 1682--1693.

Digital Library

Cited By

Reitmaier TRaju DKlejch OWallington EMarkl NPearson JJones MBell PRobinson S(2024)Cultivating Spoken Language Technologies for Unwritten LanguagesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642026(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642026
Galphat YVaswani BGangwani CDhekale S(2024)EmoSpeak: An Emotionally Intelligent Text-to-Speech System for Visually Impaired2024 International Conference on Advancements in Power, Communication and Intelligent Systems (APCI)10.1109/APCI61480.2024.10616666(1-6)Online publication date: 21-Jun-2024
https://doi.org/10.1109/APCI61480.2024.10616666
Reitmaier TWallington EKlejch OMarkl NLam-Yee-Mui LPearson JJones MBell PRobinson S(2023)Situating Automatic Speech Recognition Development within Communities of Under-heard Language SpeakersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581385(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581385
Show More Cited By

Index Terms

BSpeak: An Accessible Voice-based Crowdsourcing Marketplace for Low-Income Blind People
1. Human-centered computing

Recommendations

Respeak: A Voice-based, Crowd-powered Speech Transcription System
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Speech transcription is an expensive service with high turnaround time for audio files containing languages spoken in developing countries and regional accents of well-represented languages. We present Respeak - a voice-based, crowd-powered system that ...
Understanding Motivations and Barriers to Exercise Among People with Blindness in India
Human-Computer Interaction – INTERACT 2021
Abstract
People with vision impairments (PVIs) have inferior physical fitness compared to sighted individuals. Several studies have been conducted to understand concerns, motivations, barriers, and experiences with exercise in the blind population. However,...
Advocacy as Access Work: How People with Visual Impairments Gain Access to Digital Banking in India
CSCW

Research in accessibility and assistive technology often assumes that technology is within easy reach, that is, people with disabilities are able to obtain technologies so long as they are accessible. As a result, less is understood about the challenges ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
598
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)6

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Reitmaier TRaju DKlejch OWallington EMarkl NPearson JJones MBell PRobinson S(2024)Cultivating Spoken Language Technologies for Unwritten LanguagesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642026(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642026
Galphat YVaswani BGangwani CDhekale S(2024)EmoSpeak: An Emotionally Intelligent Text-to-Speech System for Visually Impaired2024 International Conference on Advancements in Power, Communication and Intelligent Systems (APCI)10.1109/APCI61480.2024.10616666(1-6)Online publication date: 21-Jun-2024
https://doi.org/10.1109/APCI61480.2024.10616666
Reitmaier TWallington EKlejch OMarkl NLam-Yee-Mui LPearson JJones MBell PRobinson S(2023)Situating Automatic Speech Recognition Development within Communities of Under-heard Language SpeakersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581385(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581385
Luo YLee BKim YChoe E(2022)NoteWordy: Investigating Touch and Speech Input on Smartphones for Personal Data CaptureProceedings of the ACM on Human-Computer Interaction10.1145/35677346:ISS(568-591)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567734
Rechkemmer AYin M(2022)Understanding the Microtask Crowdsourcing Experience for Workers with Disabilities: A Comparative ViewProceedings of the ACM on Human-Computer Interaction10.1145/35551376:CSCW2(1-30)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555137
Reitmaier TWallington EKalarikalayil Raju DKlejch OPearson JJones MBell PRobinson S(2022)Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language SpeakersProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517639(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517639
Varanasi RSiddarth DSeshadri VBali KVashistha A(2022)Feeling Proud, Feeling Embarrassed: Experiences of Low-income Women with Crowd WorkProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501834(1-18)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501834
Rothschild ABooker JDavoll CHill JIvey VDiSalvo CRydal Shapiro BDiSalvo B(2022)Towards fair and pro-social employment of digital pieceworkers for sourcing machine learning training dataExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3516384(1-9)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3516384
Arakawa RYakura HGoto M(2022)BeParrot: Efficient Interface for Transcribing Unclear Speech via RespeakingProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511164(832-840)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1145/3490099.3511164
Paulino DCorreia AGuimaraes DBarroso JParedes H(2022)Uncovering the Potential of Cognitive Personalization for UI Adaptation in Crowd Work2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD54268.2022.9776164(484-489)Online publication date: 4-May-2022
https://doi.org/10.1109/CSCWD54268.2022.9776164
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents