research-article

The Crowd is a Collaborative Network

Authors:

Siddharth Suri,

Syed Shoaib Ali, and

Deepti KulkarniAuthors Info & Claims

CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

February 2016

Pages 134 - 147

https://doi.org/10.1145/2818048.2819942

Published: 27 February 2016 Publication History

Abstract

The main goal of this paper is to show that crowdworkers collaborate to fulfill technical and social needs left by the platform they work on. That is, crowdworkers are not the independent, autonomous workers they are often assumed to be, but instead work within a social network of other crowdworkers. Crowdworkers collaborate with members of their networks to 1) manage the administrative overhead associated with crowdwork, 2) find lucrative tasks and reputable employers and 3) recreate the social connections and support often associated with brick and mortar-work environments. Our evidence combines ethnography, interviews, survey data and larger scale data analysis from four crowdsourcing platforms, emphasizing the qualitative data from the Amazon Mechanical Turk (MTurk) platform and Microsoft's proprietary crowdsourcing platform, the Universal Human Relevance System (UHRS). This paper draws from an ongoing, longitudinal study of Crowdwork that uses a mixed methods approach to understand the cultural meaning, political implications, and ethical demands of crowdsourcing.

References

[1]

Andrew Abbott. 2005. The sociology of work and occupations. In Handbook on Economic Sociology, Neil J. Smelser and Richard Swedberg (eds.). Russell Sage Foundation and Princeton University Press, New York and Princedon, 307-330.

[2]

Andrew Abbott. 1993. The sociology of work and occupations. Annu Rev Sociol 19: 187-209.

[3]

Amara – Caption, Translate, Subtitle and Transcribe Video. 2015. Retrieved May 21, 2015 from http://www.amara.org

[4]

Vashi Ambati, Stephan Vogel, Jamie Carbonell. 2012. Collaborative workflow for crowdsourcing translation. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '12), 1191-1194.

Digital Library

[5]

Morgan G. Ames, Janet Go, Joseph Kaye, Mirjana Spasojevic. 2010. Making love in the network closet: The benefits and work of family videochat. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '10), 145-154.

Digital Library

[6]

Benjamin B. Bederson, Alexander J. Quinn. 2011. Web workers unite! Addressing challenges of online laborers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 97105.

Digital Library

[7]

Michael S. Bernstein, Desney Tan, Greg Smith, Mary Czerwinski, Eric Horvitz. 2010. Personalization via friendsourcing. ACM Trans Comput-Hum Interact 17, 2: 1-28.

Digital Library

[8]

Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C. Miller, Aubrey Tatarowicz, Brandyn White, Samuel White, Tom Yeh. 2010. Nearly Real-time answers to visual questions. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '10), 1-10.

Digital Library

[9]

Daren C. Brabham. 2014. Moving the crowd at Threadless. Information, Communication, & Society 13 (8): 1122-1145.

[10]

Jon Chamberlain. 2014. Harnessing the intelligence of the crowd for problem solving and knowledge discovery In Proceedings on the AAAI Conference on Human Computation and Crowdsourcing (HCOMP '14), 1-5.

[11]

Lydia B. Chilton, John J. Horton, Robert C. Miller, Shiri Azenkot. 2010. Task search in a human computation market. In Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining (KIGKDD '10), 1-9.

Digital Library

[12]

Lydia B. Chilton, Juho Kim, Paul André, Felicia Cordeiro, James A. Landay, Daniel S. Weld, Steven P. Dow, Robert C. Miller, Haoqi Zhang. 2014. Frenzy: collaborative data organization for creating conference sessions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1-10.

Digital Library

[13]

Franco Curmi, Maria Angela Ferrario, John Whittle, Floyd Mueller. 2015. Crowdsourcing synchronous spectator support: (go on, go on, you're the best). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15), 757-766.

Digital Library

[14]

Alison Davis-Blake, Joseph P. Broschak. 2009. Outsourcing and the changing nature of work. Annu Rev Socio 35: 321-340.

[15]

Paul Dourish. 1995. Developing a reflective model of collaborative systems. ACM Trans Comput-Hum Interact 2, 1: 40-63.

Digital Library

[16]

Elizabeth Gerber, Julie Hui. 2013. . Crowdfunding: Motivations and deterrents for participation. ACM Trans Comput-Hum Interact 20, 6: 1-32.

Digital Library

[17]

Elizabeth Gerber, Michael Muller, Rick Wash, Elizabeth F. Churchill, Lily C. Arani, Amanda Williams. 2014. Crowdfunding: An emerging field of research. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1093-1098.

Digital Library

[18]

Jonathan Grudin. 1994. Computer-supported cooperative work: History and focus. Comput 27, 5: 19-26.

Digital Library

[19]

Neha Gupta, David Martin, Ben Hanrahan, Jacki O'Neill. 2014. Turk-life in India. In Proceedings on the ACM International Conference on Supporting Group Work (GROUP '14), 1-11.

Digital Library

[20]

Vaughn Hester, Aaron Shaw, Lucas Biewald. 2010. Scalable crisis relief: Crowdsourced SMS translation and categorization with Mission 4636. In Proceedings of the Symposium on Computing for Development (DEV '10), 1-7.

Digital Library

[21]

James Hollan, Edwin Hutchins, David Kirsh. Distributed cognition: toward a new foundation for human-computer interaction research. ACM Trans Comput-Hum Interact 7, 2: 174-196.

Digital Library

[22]

Jeff Howe. 2006. The rise of crowdsourcing. Wired Magazine 14: 1-4.

[23]

Shih-Wen Huang, Wan-Tat Fu. 2013. Don't hide in the crowd! Increasing social transparency between peer workers improves crowdsourcing outcomes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 621-630.

Digital Library

[24]

Shih-Wen Huang, Wan-Tat Fu. 2013. Motivating crowds using social facilitation and social transparency. In CSCW '13 Companion, 149-152.

Digital Library

[25]

Lily Irani, M. Six Silberman. 2013. Turkopticon: Interrupting worker invisibility in Amazon Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1-10.

Digital Library

[26]

Arne L. Kalleberg. Nonstandard employment relations: Part-time, temporary and contract work. Annu Rev Sociol 26: 341-365.

[27]

Aniket Kittur, Ed Chi, Bryan A. Pendleton, Bongwon Suh, Todd Mytkowicz. 2009. Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 19.

[28]

Aniket Kittur, Robert E. Kraut. Harnessing the wisdom of crowds in Wikipedia: Quality through coordination. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '08), 37-46.

Digital Library

[29]

Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, John Horton. 2013. The future of crowd work. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '13), 1301-1317.

Digital Library

[30]

Anand Kulkarni, Matthew Can, Björn Hartmann. 2012. Collaboratively crowdsourcing workflows with Turkomatic. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '12), 1003-1012.

Digital Library

[31]

LeadGenius – B2B Lead Generation and Custom Data. 2015. Retrieved May 21, 2015 from http://www.leadgenius.com

[32]

Charlotte P. Lee, Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA): A Conceptual Framework of and for CSCW. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '15), 179-194.

Digital Library

[33]

Greg Little, Lydia B. Chilton, Robert C. Miller, Max Goldman. 2009. TurKit: Tools for iteratives on Mechanical Turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP '09), 29-30.

Digital Library

[34]

Munir Mandviwalla, Lorne Olfman. 1994. What do groups need? A proposed set of generic groupware requirements. ACM Trans Comput-Hum Interact 1, 3: 245-268.

Digital Library

[35]

David Martin, Benjamin V. Hanrahan, Jacki O'Neill, Neha Gupta. 2014. Being a turker. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '14), 1-12.

Digital Library

[36]

Lev Muchnik, Sinan Aral, Sean J. Taylor. 2013. Social influence bias: A randomized experiment. Science 341, 6146: 647-651.

[37]

Jacki O'Neill, David Martin. 2013. Relationship-based business process crowdsourcing. In Proceedings of the IFIP conference on human-computer interaction (INTERACT '13), 429-446.

[38]

Cheong Ha Park, KyongHee Son, Joon Hyub Lee, Seok-Hyung Bae. 2013. Crowd vs. crowd: Large-scale cooperative design through open team competition. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '13).

Digital Library

[39]

Niloufar Salehi, Lilly C. Irani, Michael S. Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milland, Clickhappier. 2015. We are dynamo: Overcoming stalling and friction in collective action for crowd workers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15), 1-10.

Digital Library

[40]

Aaron Shaw, Benjamin Mako Hill. 2014. Laboratories of oligarchy? How the iron law extends to peer production. J Commun 64, 2: 214-238.

[41]

Aaron Shaw, Haoqi Zhang, Andrés MonroyHernández, Sean Munson, Benjamin Hill, Elizabeth Gerber, Peter Kinnaird, Patrick Minder. 2014. Computer supported collective action. ACM Interactions 24, 2: 74-77.

Digital Library

[42]

M. Six Silberman, Joel Ross, Lily Irani, Bill Tomlinson. 2010. Sellers' problems in human computation markets. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP '10), 18-21.

Digital Library

[43]

Vicki Smith. 1997. New forms of work organization. Annu Rev Sociol 23: 315-339.

[44]

Rion Snow, Brendan O'Connor, Daniel Jurafsky, Andrew Y. Ng. 2008. Cheap and fast—but is it good? Evaluating non-expert annotations for natural language tasks. In Proceedings of the ACL Conference on Empirical Methods in Natural Language Processing (EMNLP '08), 254-263.

Digital Library

[45]

Anslem Strauss. 1988. The articulation of project work: An organizational process. Sociological Quarterly 29, 2: 163-178.

[46]

James Surowiecki. 2005. The wisdom of crowds. Random House.

Digital Library

[47]

Haoqi Zhang, Eric Horvitz, Rob C. Miller, and David C. Parkes. 2011. Crowdsourcing general computation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 1-4.

[48]

Haiyi Zhu, Robert Kraut, Aniket Kittur. 2012. Group identification, goal setting, and social modeling in directing online production. In Proceedings of the ACM conference on Computer supported cooperative work (CSCW '12), 935-944.

Digital Library

[49]

Vinko Zlatić, Miran Božičević, Hrvoje Štefančić, Mladen Domazet. 2006. Wikipedias: Collaborative web-based encyclopedias as complex networks. Phys Rev E 74, 1.

Cited By

Jonas A(2024)'Not Just a Computer Screen:' Distributed Attunement in Virtual School EnvironmentsProceedings of the ACM on Human-Computer Interaction10.1145/36410128:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641012
Pendse SMassachi TMahdavimoghaddam JButler JSuh JCzerwinski M(2024)Towards Inclusive Futures for Worker WellbeingProceedings of the ACM on Human-Computer Interaction10.1145/36374148:CSCW1(1-32)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637414
Fulker ZRiedl C(2024)Cooperation in the Gig Economy: Insights from Upwork FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/36373148:CSCW1(1-20)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637314
Show More Cited By

The Crowd is a Collaborative Network
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
2. Social and professional topics
  1. Professional topics
    1. Computing and business

Recommendations

The Communication Network Within the Crowd
WWW '16: Proceedings of the 25th International Conference on World Wide Web

Since its inception, crowdsourcing has been considered a black-box approach to solicit labor from a crowd of workers. Furthermore, the "crowd" has been viewed as a group of independent workers dispersed all over the world. Recent studies based on in-...
Read More
Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments

The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that ...
Read More
Efficient Crowd Exploration of Large Networks: The Case of Causal Attribution

Accurately and efficiently crowdsourcing complex, open-ended tasks can be difficult, as crowd participants tend to favor short, repetitive "microtasks". We study the crowdsourcing of large networks where the crowd provides the network topology via ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

February 2016

1866 pages

ISBN:9781450335928

DOI:10.1145/2818048

General Chairs:
Darren Gergle
Northwestern University
,
Meredith Ringel Morris
Microsoft Research
,
Program Chairs:
Pernille Bjørn
University of Copenhagen
,
Joseph Konstan
University of Minnesota

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 February 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CSCW '16

Sponsor:

SIGCHI

CSCW '16: Computer Supported Cooperative Work and Social Computing

February 27 - March 2, 2016

California, San Francisco, USA

Acceptance Rates

CSCW '16 Paper Acceptance Rate 142 of 571 submissions, 25%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '24

Sponsor:
sigchi

CSCW '24: Computer-Supported Cooperative Work and Social Computing

November 9 - 13, 2024

San Jose , Costa Rica

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

137
Total Citations
View Citations
727
Total Downloads

Downloads (Last 12 months)85
Downloads (Last 6 weeks)8

Other Metrics

View Author Metrics

Citations

Cited By

Jonas A(2024)'Not Just a Computer Screen:' Distributed Attunement in Virtual School EnvironmentsProceedings of the ACM on Human-Computer Interaction10.1145/36410128:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641012
Pendse SMassachi TMahdavimoghaddam JButler JSuh JCzerwinski M(2024)Towards Inclusive Futures for Worker WellbeingProceedings of the ACM on Human-Computer Interaction10.1145/36374148:CSCW1(1-32)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637414
Fulker ZRiedl C(2024)Cooperation in the Gig Economy: Insights from Upwork FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/36373148:CSCW1(1-20)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637314
de Groot EGadiraju U(2024)"Are we all in the same boat?" Customizable and Evolving Avatars to Improve Worker Engagement and Foster a Sense of Community in Online Crowd WorkProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642429(1-26)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642429
Banerjee D(2024)Caste, gig-economy, and youth: the resurgence of a new age caste politics in urban GujaratSouth Asian History and Culture10.1080/19472498.2024.234686715:2(231-246)Online publication date: 2-May-2024
https://doi.org/10.1080/19472498.2024.2346867
Li NWang BLi YPang HHan X(2024)Beyond the Swipe: Investigating the Interplay of Technology, Media, and Human Behavior in Digital RomanceJournal of the Knowledge Economy10.1007/s13132-024-01802-6Online publication date: 5-Feb-2024
https://doi.org/10.1007/s13132-024-01802-6
Shaikh RMcGregor MBrown BLampinen A(2024)Fleeting Alliances and Frugal Collaboration in Piecework: A Video-Analysis of Food Delivery Work in IndiaComputer Supported Cooperative Work (CSCW)10.1007/s10606-024-09501-1Online publication date: 21-Jun-2024
https://doi.org/10.1007/s10606-024-09501-1
Chintala S(2023)Emerging Patterns of Collective Action in India’s Platform Economy: A Study of Drivers and Delivery WorkersRelations industrielles10.7202/1110483ar78:3Online publication date: 2023
https://doi.org/10.7202/1110483ar
Ghita R(2023)Do mturkers collude in interactive online experiments?Behavior Research Methods10.3758/s13428-023-02220-3Online publication date: 1-Sep-2023
https://doi.org/10.3758/s13428-023-02220-3
Becker AEcker LKülpmann ISchwien KStobbe P(2023)Cooperative solidarity among crowdworkers? Social learning practices on a crowdtesting social media platformConvergence: The International Journal of Research into New Media Technologies10.1177/1354856523118329830:1(428-449)Online publication date: 15-Jun-2023
https://doi.org/10.1177/13548565231183298
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents