Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2872427.2883036acmotherconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

The Communication Network Within the Crowd

Published: 11 April 2016 Publication History

Abstract

Since its inception, crowdsourcing has been considered a black-box approach to solicit labor from a crowd of workers. Furthermore, the "crowd" has been viewed as a group of independent workers dispersed all over the world. Recent studies based on in-person interviews have opened up the black box and shown that the crowd is not a collection of independent workers, but instead that workers communicate and collaborate with each other. Put another way, prior work has shown the existence of edges between workers. We build on and extend this discovery by mapping the entire communication network of workers on Amazon Mechanical Turk, a leading crowdsourcing platform. We execute a task in which over 10,000 workers from across the globe self-report their communication links to other workers, thereby mapping the communication network among workers. Our results suggest that while a large percentage of workers indeed appear to be independent, there is a rich network topology over the rest of the population. That is, there is a substantial communication network within the crowd. We further examine how online forum usage relates to network topology, how workers communicate with each other via this network, how workers' experience levels relate to their network positions, and how U.S. workers differ from international workers in their network characteristics. We conclude by discussing the implications of our findings for requesters, workers, and platform providers like Amazon.

References

[1]
S. P. Borgatti and M. G. Everett. Models of core/periphery structures. Social Networks, 21: 375--395, 1999.
[2]
R. S. Burt. Structural holes and good ideas. American Journal of Sociology, 110 (2): 349--99, September 2004.
[3]
R. S. Burt. Second-hand brokerage: Evidence on the importance of local structure for managers, bankers, and analysts. Academy of Management Journal, 50: 119--148, 2007.
[4]
S. Currarini, M. O. Jackson, and P. Pin. An economic model of friendship: Homophily, minorities, and segregation. Econometrica, 77: 1003--1045, 2009.
[5]
D. Easley and J. Kleinberg. Networks, crowds, and markets: Reasoning about a highly connected world. Cambridge University Press, 2010.
[6]
M. L. Gray, S. Suri, S. S. Ali, and D. Kulkarni. The crowd is a collaborative network. In The 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW), 2016.
[7]
N. Gupta, D. Martin, B. V. Hanrahan, and J. O'Neil. Turk-life in India. In The Intnernational Conference on Supporting Groupwork (Group), 2014.
[8]
L. C. Irani and M. S. Silberman. Turkopticon: Interrupting worker invisibility in Amazon Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2013.
[9]
J. Kleinberg, S. Suri, Éva Tardos, and T. Wexler. Strategic network formation with structural holes. In Proceedings of the 9th ACM Conference on Electronic Commerce (EC), 2008.
[10]
D. Martin, B. V. Hanrahan, J. O'Neil, and N. Gupta. Being a turker. In The 17th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW), 2014.
[11]
W. Mason and S. Suri. Conducting behavioral research on Amazon's Mechanical Turk. Behavior Research Methods, 44 (1): 1--23, March 2012.
[12]
M. McPherson, L. Smith-Lovin, and J. M. Cook. Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27: 415--444, 2001.
[13]
G. Paolacci, J. Chandler, and P. G. Ipeirotis. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5: 411--419, 2010.
[14]
D. G. Rand. The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments. Journal of Theoretical Biology, 299: 172--179, April 2011.
[15]
N. Salehi, L. C. Irani, M. S. Bernstein, A. Alkhatib, E. Ogbe, K. Milland, and Clickhappier. We are Dynamo: Overcoming stalling and friction in collective action for crowd workers. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), 2015.
[16]
N. Stewart, C. Ungemach, A. J. L. Harris, D. M. Bartels, B. R. Newell, G. Paolacci, and J. Chandler. The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, September 2015.
[17]
S. Wasserman and K. Faust. Social network analysis: Methods and applications, volume 8. Cambridge University Press, 1994.
[18]
D. J. Watts and S. H. Strogatz. Collective dynamics of "small-world" networks. Nature, 393 (6684): 440--442, 1998.
[19]
K. Zyskowksi and K. Milland. Crowdworking visibility: An ethnography of a discussion board for digital labor. Unpublished Manuscript, 2015.

Cited By

View all
  • (2024)Work Schedules, Finances, and Freedom: Work Schedule Fit and Platform Dependence among Gig Workers on Amazon’s Mechanical Turk PlatformSocius: Sociological Research for a Dynamic World10.1177/2378023124128693310Online publication date: 17-Oct-2024
  • (2024)Deeply embedded wages: Navigating digital payments in data workBig Data & Society10.1177/2053951724124244611:2Online publication date: 30-Apr-2024
  • (2024)"Guilds" as Worker Empowerment and Control in a Chinese Data Work PlatformProceedings of the ACM on Human-Computer Interaction10.1145/36869048:CSCW2(1-27)Online publication date: 8-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
WWW '16: Proceedings of the 25th International Conference on World Wide Web
April 2016
1482 pages
ISBN:9781450341431

Sponsors

  • IW3C2: International World Wide Web Conference Committee

In-Cooperation

Publisher

International World Wide Web Conferences Steering Committee

Republic and Canton of Geneva, Switzerland

Publication History

Published: 11 April 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. crowdsourcing
  2. mechanical turk
  3. networks
  4. online forums

Qualifiers

  • Research-article

Conference

WWW '16
Sponsor:
  • IW3C2
WWW '16: 25th International World Wide Web Conference
April 11 - 15, 2016
Québec, Montréal, Canada

Acceptance Rates

WWW '16 Paper Acceptance Rate 115 of 727 submissions, 16%;
Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)29
  • Downloads (Last 6 weeks)4
Reflects downloads up to 08 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Work Schedules, Finances, and Freedom: Work Schedule Fit and Platform Dependence among Gig Workers on Amazon’s Mechanical Turk PlatformSocius: Sociological Research for a Dynamic World10.1177/2378023124128693310Online publication date: 17-Oct-2024
  • (2024)Deeply embedded wages: Navigating digital payments in data workBig Data & Society10.1177/2053951724124244611:2Online publication date: 30-Apr-2024
  • (2024)"Guilds" as Worker Empowerment and Control in a Chinese Data Work PlatformProceedings of the ACM on Human-Computer Interaction10.1145/36869048:CSCW2(1-27)Online publication date: 8-Nov-2024
  • (2024)The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and GuidelinesProceedings of the ACM on Human-Computer Interaction10.1145/36410238:CSCW1(1-45)Online publication date: 26-Apr-2024
  • (2024)"Are we all in the same boat?" Customizable and Evolving Avatars to Improve Worker Engagement and Foster a Sense of Community in Online Crowd WorkProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642429(1-26)Online publication date: 11-May-2024
  • (2024)The Use of AI-powered Language Tools in Crowdsourcing to reduce Language Barriers2024 IEEE International Conferences on Internet of Things (iThings) and IEEE Green Computing & Communications (GreenCom) and IEEE Cyber, Physical & Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics62450.2024.00110(601-608)Online publication date: 19-Aug-2024
  • (2023)Testing conventional wisdom (of the crowd)Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence10.5555/3625834.3625857(237-248)Online publication date: 31-Jul-2023
  • (2023)Do mturkers collude in interactive online experiments?Behavior Research Methods10.3758/s13428-023-02220-356:5(4823-4835)Online publication date: 1-Sep-2023
  • (2023)Preference-Matched Multitask Assignment for Group Socialization under Mobile CrowdsensingSensors10.3390/s2304227523:4(2275)Online publication date: 17-Feb-2023
  • (2023)Cooperative solidarity among crowdworkers? Social learning practices on a crowdtesting social media platformConvergence: The International Journal of Research into New Media Technologies10.1177/1354856523118329830:1(428-449)Online publication date: 15-Jun-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media