research-article

Investigating Crowdworkers' Identify, Perception and Practices in Micro-Task Crowdsourcing

Authors:

Kai Ye, and

Liang HeAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue GROUP

Article No.: 35, Pages 1 - 20

https://doi.org/10.1145/3492854

Published: 14 January 2022 Publication History

Abstract

Crowdsourcing is rapidly gaining popularity among academic and business communities. Yet, our understanding of this work way is still in its incipient stage, in particular regarding the increasingly large and diverse crowdworkers. As such, we aim to understand crowdworkers' perception and experience to themselves and their work from their own perspective. We explore this by a mix-methods study of crowdworkers in Ali, one of prominent micro-task crowdsourcing platforms in China. Our findings highlight crowdworker in Ali is not only a coded name, but also an identity with some positive attitudes and beliefs towards work and life. In particular, this identity provides many socio-psychological benefits for crowdworkers, which further contributes to their consistent engagement in Ali and proactive practices to improve crowdworker communities and Ali platform collaboratively. We according suggest that taking crowdworker identity as a lens for crowdsourcing research, and turning attention towards construction and expressions of crowdworkers' identity and values in their own context.

References

[1]

Top 5 prominent crowdsourcing platforms in China, https://baijiahao.baidu.com/s?id=1668289901442729625&wfr=spider&for=pc

[2]

Ali Crowdsourcing, https://www.aliyun.com/product/aicrowd/.

[3]

Liliyai Besaleva and Alfredc Weaver. CrowdHelp: A crowdsourcing application for improving disaster management. In Proceedings of 2013 IEEE Global Humanitarian Technology Conference (GHTC), 2013, 185--190, https://doi.org/10.1109/GHTC.2013.6713678.

[4]

Liliyai Besaleva and Alfredc Weaver. CrowdHelp: application for improved emergency response through crowdsourced information. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, 2013, 1437--1446, https://doi.org/10.1145/2494091.2496040.

Digital Library

[5]

Darenc Brabham. Crowdsourcing as a model for problem solving: An introduction and cases. Convergence 14, 1 (2008), 75--90.

[6]

Stacym Branham, Anja Thieme, Lisap Nathan, Steve Harrison and Deborah Tatar, et al. Co-creating and identity-making in CSCW: revisiting ethics in design research. In Proceedings of the companion publication of the 17th ACM conference on Computer supported cooperative work and social computing (CSCW 2014), 2014, 305--308, https://doi.org/10.1145/2556420.2558859.

Digital Library

[7]

D. Bratvold. Enterprise crowdsourcing blasts off as social media growth industry. Schaefer Marketing (2011).

[8]

Andrewd Brown. Identities and identity work in organizations. INTERNATIONAL JOURNAL OF MANAGEMENT REVIEWS 17, 1 (2015), 20--40.

[9]

Ang Cao and Pengxiang Li. We are not machines: The identity construction of Chinese female migrant workers in online chat groups. Chinese Journal of Communication 11, 3 (2018), 289--305, https://doi.org/10.1080/17544750.2018.1435555.

[10]

Josephchee Chang, Saleema Amershi and Ece Kamar. Revolt: Collaborative crowdsourcing for labeling machine learning datasets. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI 2017), 2017, 2334--2346, https://doi.org/10.1145/3025453.3026044.

Digital Library

[11]

Kathy Charmaz and Lindaliska Belgrave. Grounded theory. The Blackwell encyclopedia of sociology (2007).

[12]

Gilbert Cockton. Value-centred HCI. In Proceedings of the third Nordic conference on Human-computer interaction (NordiCHI 2004), 2004, 149--160, https://doi.org/10.1145/1028014.1028038.

Digital Library

[13]

Xuefei Deng and Kshitid Joshi. Why individuals participate in micro-task crowdsourcing work environment: Revealing crowdworkers' perceptions. Journal of the Association for Information Systems 17, 10 (2016), 648.

[14]

Djellel Difallah, Elena Filatova and Panos Ipeirotis. Demographics and dynamics of mechanical turk workers. In Proceedings of the eleventh ACM international conference on web search and data mining, 2018, 135--143, https://doi.org/10.1145/3159652.3159661.

Digital Library

[15]

Saraha Donovan, Davidh Bradley and Jono Shimabukuru. What does the gig economy mean for workers? (2016).

[16]

Janee Dutton, Lauramorgan Roberts and Jeffrey Bednar. Pathways for positive identity construction at work: Four types of positive identity and the building of social resources. ACADEMY OF MANAGEMENT REVIEW 35, 2 (2010), 265--293.

[17]

Naomi Ellemers, Russell Spears and Bertjan Doosje. Self and social identity. Annual Review of Psychology 53, 1 (2002), 161--186.

[18]

Erikhomburger Erikso. Identity and the life cycle: Selected papers. (1959).

[19]

Ivan Fedorenko, Pierre Berthon and Tamara Rabinovich. Crowded identity: Managing crowdsourcing initiatives to maximize value for participants through identity creation. BUSINESS HORIZONS 60, 2 (2017), 155--165.

[20]

Rimantas Gatautis and Elena Vitkauskaite. Crowdsourcing application in marketing activities. Procedia-Social and Behavioral Sciences 110 (2014), 1243--1250.

[21]

Maryl Gray, Siddharth Suri, Syedshoaib Ali and Deepti Kulkarni. The crowd is a collaborative network. In Proceedings of the 19th ACM conference on computer-supported cooperative work and social computing (CSCW 2016), 2016, 134--147, https://doi.org/10.1145/2818048.2819942.

Digital Library

[22]

Neha Gupta, David Martin, Benjaminv Hanrahan and Jacki O'Neill. Turk-life in India. In Proceedings of the 18th International Conference on Supporting Group Work (Group 2014), 2014, 1--11, https://doi.org/10.1145/2660398.2660403.

Digital Library

[23]

Jeff Howe. The rise of crowdsourcing. Wired magazine 14, 6 (2006), 1--4.

[24]

Shih-wen Huang and Wai-tat Fu. Don't hide in the crowd!: increasing social transparency between peer workers improves crowdsourcing outcomes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), 2013, 621--630, https://doi.org/10.1145/2470654.2470743.

Digital Library

[25]

Shengping Huo, Zheng Chijian and Hailang Tu. On the Research of Intelligent Coupling and Coordination between Employers and Witkey on the Crowd Innovation Network Platform. Journal of Xiangtan University 41, 1 (2017), 94--101.

[26]

Ayomikun Idowu and Amany Elbanna. Crowdworkers, social affirmation and work identity: Rethinking dominant assumptions of crowdwork. Information and Organization (2021), 100335.

[27]

Kazushi Ikeda, and Keiichiro Hoashi. Crowdsourcing GO: Effect of Worker Situation on Mobile Crowdsourcing Performance. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI 2017), 2017, 1142--1153, https://doi.org/10.1145/3025453.3025917.

Digital Library

[28]

Lillyc Irani and M Silberman. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. In Proceedings of the SIGCHI conference on human factors in computing systems (CHI 2013), 2013, 611--620, https://doi.org/10.1145/2470654.2470742.

Digital Library

[29]

Ling Jiang, Christian Wagner and Bonnie Nardi. Not just in it for the money: a qualitative investigation of workers' perceived benefits of micro-task crowdsourcing. In Proceedings of the 48th Hawaii International Conference on System Sciences, 2015, 773--782, https://ieeexplore.ieee.org/document/7069747/.

Digital Library

[30]

Yu Jiang, Yuling Sun, Jing Yang, Xin Lin and Liang He. Enabling uneven task difficulty in micro-task crowdsourcing. In Proceedings of the 2018 ACM Conference on Supporting Groupwork (Group 2018), 2018, 12--21, https://doi.org/10.1145/3148330.3148342.

Digital Library

[31]

Yuchao Jiang, Daniel Schlagwein and Boualem Benatallah. A Review on Crowdsourcing for Education: State of the Art of Literature and Practice. In PACIS, 2018, 180, https://aisel.aisnet.org/pacis2018/180.

[32]

Nicolas Kaufmann, Thimo Schulze and Daniel Veit. More than fun and money. Worker Motivation in Crowdsourcing-A Study on Mechanical Turk. In the 17th Americas Conference on Information Systems (AMCIS), 2011, 1--11, http://aisel.aisnet.org/amcis2011_submissions/340/.

[33]

Aniket Kittur, Edh Chi and Bongwon Suh. Crowdsourcing user studies with Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2008), 2008, 453--456, https://doi.org/10.1145/1357054.1357127.

Digital Library

[34]

Aniket Kittur, Jeffreyv Nickerson, Michael Bernstein, Elizabeth Gerber and Aaron Shaw, et al. The future of crowd work. In Proceedings of the 2013 conference on Computer supported cooperative work (CSCW 2013), 2013, 1301--1318, https://doi.org/10.1145/2441776.2441923.

Digital Library

[35]

Thomasd Latoza and Andre Van Der Hoek. Crowdsourcing in software engineering: Models, motivations, and challenges. IEEE SOFTWARE 33, 1 (2015), 74--80.

[36]

Alec Lynch. Crowdsourcing is booming in Asia. TechCrunch (2012).

[37]

David Martin, Benjaminv Hanrahan, Jacki O'Neill and Neha Gupta. Being a turker. In Proceedings of the 17th ACM conference on Computer supported cooperative work and social computing (CSCW 2014), 2014, 224--235, https://doi.org/10.1145/2531602.2531663.

Digital Library

[38]

Shengying Pan, Kate Larson, Josh Bradshaw and Edith Law. Dynamic Task Allocation Algorithm for Hiring Workers that Learn. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI 2016), 2016, 3825--3831.

Digital Library

[39]

Eyal Peer, Laura Brandimarte, Sonam Samat and Alessandro Acquisti. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY 70 (2017), 153--163.

[40]

Dennis Pilz and Heiko Gewald. Does Money Matter? Motivational Factors for Participation in Paid-and Non-Profit-Crowdsourcing Communities. Wirtschaftsinformatik 37 (2013), 73--82.

[41]

Lisa Posch, Arnim Bleier, Fabian Fl O Ck and Markus Strohmaier. Characterizing the global crowd workforce: A cross-country comparison of crowdworker demographics. arXiv preprint arXiv:1812.05948 (2018).

[42]

Yuqing Ren, Robert Kraut and Sara Kiesler. Applying common identity and bond theory to design of online communities. ORGANIZATION STUDIES 28, 3 (2007), 377--408.

[43]

Joel Ross, Lilly Irani, M Silberman, Andrew Zaldivar and Bill Tomlinson. Who are the crowdworkers?: shifting demographics in mechanical turk. In, 2010, 2863--2872, https://doi.org/10.1145/1753846.1753873.

Digital Library

[44]

Dina Sabie, Samar Sabie and Syedishtiaque Ahmed. Memory through Design: Supporting Cultural Identity for Immigrants through a Paper-Based Home Drafting Tool. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, 1--16, https://doi.org/10.1145/3313831.3376636.

Digital Library

[45]

Ari Schlesinger, Wkeith Edwards and Rebeccae Grinter. Intersectional HCI: Engaging identity through gender, race, and class. In Proceedings of the 2017 CHI conference on human factors in computing systems, 2017, 5412--5427, https://doi.org/10.1145/3025453.3025766.

Digital Library

[46]

Florianalexander Schmidt. The good, the bad and the ugly: Why crowdsourcing needs ethics. In 2013 International Conference on Cloud and Green Computing, 2013, 531--535, https://doi.org/10.1109/CGC.2013.89.

Digital Library

[47]

Joseph Seering, Felicia Ng, Zheng Yao and Geoff Kaufman. Applications of social identity theory to research and design in social computing. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW'18). ACM, New York, NY, USA, 2018, https://doi.org/10.1145/3274771.

Digital Library

[48]

Shamir, Boas. Calculations, values, and identities: The sources of collectivistic work motivation. HUMAN RELATIONS 43, 4 (1990), 313--332.

[49]

Yuling Sun, Silvia Lindtner, Xianghua Ding, Tun Lu and Ning Gu. Reliving the Past & Making a Harmonious Society Today: A Study of Elderly Electronic Hackers in China. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2015), 2015, 44--55, https://doi.org/10.1145/2675133.2675195.

Digital Library

[50]

Yuling Sun, Xianghua Ding, Silvia Lindtner, Tun Lu and Ning Gu. Being Senior and ICT: A Study of Seniors Using ICT in China. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2014), 2014, 3933--3942, https://doi.org/10.1145/2556288.2557248.

Digital Library

[51]

Deborahj Terry, Michaela Hogg and Katherinem White. The theory of planned behaviour: self-identity, social identity and group norms. BRITISH JOURNAL OF SOCIAL PSYCHOLOGY 38, 3 (1999), 225--244.

[52]

Wai-ming To and Lindasl Lai. Crowdsourcing in China: Opportunities and concerns. IT Professional 17, 3 (2015), 53--59.

[53]

Luis Von Ahn, Manuel Blum, Nicholasj Hopper and John Langford. CAPTCHA: Using hard AI problems for security. In Proceedings of International conference on the theory and applications of cryptographic techniques, 2003, 294--311, https://doi.org/10.1007/3--540--39200--9_18.

[54]

Yihong Wang, Konstantinos Papangelis, Ioanna Lykourentzou and Vassilis-javed Khan. The changing landscape of crowdsourcing in China: from individual crowdworkers to crowdfarms. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing, 2019, 413--417, https://doi.org/10.1145/3311957.3359469.

Digital Library

[55]

Yihong Wang, Konstantinos Papangelis, Michael Saker, Ioanna Lykourentzou and Alan Chamberlain, et al. Crowdsourcing in China: Exploring the Work Experiences of Solo Crowdworkers and Crowdfarm Workers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, 1--13, https://doi.org/10.1145/3313831.3376473.

Digital Library

[56]

Paul Whitla. Crowdsourcing and its application in marketing activities. Contemporary Management Research 5, 1 (2009).

Cited By

Sun WSrinivasan K(2024)RFDrive: Tagged Human-Vehicle Interaction for AllACM Journal on Computing and Sustainable Societies10.1145/36485332:2(1-19)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3648533
Bostandoost RLechowicz AHanafy WBashir NShenoy PHajiesmaili M(2024)LACS: Learning-Augmented Algorithms for Carbon-Aware Resource Scaling with Uncertain DemandProceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems10.1145/3632775.3661942(27-45)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3632775.3661942
Tabatabaie MHe S(2024)Driver Maneuver Interaction Identification with Anomaly-Aware Federated Learning on Heterogeneous Feature RepresentationsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314217:4(1-28)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631421
Show More Cited By

Index Terms

Investigating Crowdworkers' Identify, Perception and Practices in Micro-Task Crowdsourcing
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

The Dark Side of Recruitment in Crowdsourcing: Ethics and Transparency in Micro-Task Marketplaces
Abstract
Micro-task crowdsourcing marketplaces like Figure Eight (F8) connect a large pool of workers to employers through a single online platform, by aggregating multiple crowdsourcing platforms (channels) under a unique system. This paper investigates ...
Read More
Enabling Uneven Task Difficulty in Micro-Task Crowdsourcing
GROUP '18: Proceedings of the 2018 ACM International Conference on Supporting Group Work

In micro-task crowdsourcing markets such as Amazon's Mechanical Turk, how to obtain high quality result without exceeding the limited budgets is one main challenge. The existing theory and practice of crowdsourcing suggests that uneven task difficulty ...
Read More
The Dynamics of Micro-Task Crowdsourcing: The Case of Amazon MTurk
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

Micro-task crowdsourcing is rapidly gaining popularity among research communities and businesses as a means to leverage Human Computation in their daily operations. Unlike any other service, a crowdsourcing platform is in fact a marketplace subject to ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue GROUP

GROUP

January 2022

992 pages

EISSN:2573-0142

DOI:10.1145/3511803

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 January 2022

Published in PACMHCI Volume 6, Issue GROUP

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Open Research Program of Shanghai Key Lab of Intelligent Information Processing

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)2

Other Metrics

View Author Metrics

Citations

Cited By

Sun WSrinivasan K(2024)RFDrive: Tagged Human-Vehicle Interaction for AllACM Journal on Computing and Sustainable Societies10.1145/36485332:2(1-19)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3648533
Bostandoost RLechowicz AHanafy WBashir NShenoy PHajiesmaili M(2024)LACS: Learning-Augmented Algorithms for Carbon-Aware Resource Scaling with Uncertain DemandProceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems10.1145/3632775.3661942(27-45)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3632775.3661942
Tabatabaie MHe S(2024)Driver Maneuver Interaction Identification with Anomaly-Aware Federated Learning on Heterogeneous Feature RepresentationsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314217:4(1-28)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631421
Khan HAli FGhadi YNazir SUllah IMohamed H(2023)Human–Computer Interaction and Participation in Software CrowdsourcingElectronics10.3390/electronics1204093412:4(934)Online publication date: 13-Feb-2023
https://doi.org/10.3390/electronics12040934
Correia AGrover ASchneider DPimentel AChaves Rde Almeida MFonseca B(2023)Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine InteractionApplied Sciences10.3390/app1304219813:4(2198)Online publication date: 8-Feb-2023
https://doi.org/10.3390/app13042198
Lechowicz AChristianson NZuo JBashir NHajiesmaili MWierman AShenoy P(2023)The Online Pause and Resume Problem: Optimal Algorithms and An Application to Carbon-Aware Load ShiftingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/36267767:3(1-32)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1145/3626776
Hajiesmaili M(2023)Trade-off Analysis in Learning-augmented Algorithms with Societal Design CriteriaACM SIGMETRICS Performance Evaluation Review10.1145/3626570.362659051:2(53-58)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3626570.3626590
Sun BYang LHajiesmaili MWierman ALui JTowsley DTsang D(2023)The Online Knapsack Problem with DeparturesACM SIGMETRICS Performance Evaluation Review10.1145/3606376.359357651:1(59-60)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1145/3606376.3593576
Rother ASpiliopoulou M(2022)Virtual Reality for Medical Annotation Tasks: A Systematic ReviewFrontiers in Virtual Reality10.3389/frvir.2022.7173833Online publication date: 29-Jun-2022
https://doi.org/10.3389/frvir.2022.717383

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents