research-article

Workflow transparency in a microtask marketplace

Authors:

Peter Kinnaird,

Sara KieslerAuthors Info & Claims

GROUP '12: Proceedings of the 2012 ACM International Conference on Supporting Group Work

Pages 281 - 284

https://doi.org/10.1145/2389176.2389219

Published: 27 October 2012 Publication History

Abstract

Interdependent tasks in Mechanical Turk (MTurk) can be managed efficiently with a workflow, a sequence of tasks through which work passes to its completion. We ask if workers should be informed about the workflow, which we call workflow transparency. Transparency could motivate workers or induce social loafing. We describe three experiments to determine the effects of workflow transparency in MTurk. We compared a text description of the workflow, a visualization of the workflow, and the combination of text and visualization with a control condition giving no workflow information. Workflow transparency marginally increased volunteerism on a charity identification task (experiment 1) and significantly increased volunteerism and quality on a business identification task (experiment 2). Results were weaker with a less experienced worker sample (experiment 3). We suggest further research on the design of workflow information to increase workers' motivation.

References

[1]

Bernstein, M. S. et al. 2010. Soylent: a word processor with a crowd inside. UIST (2010), 313--322.

Digital Library

[2]

Downs, J. S. et al. 2010. Are Your Participants Gaming the System? Screening Mechanical Turk Workers. Science. (2010), 2399--2402.

Digital Library

[3]

Fekete, J.-D. et al. 2008. Information Visualization. A. Kerren et al., eds. Springer-Verlag. 1--18.

[4]

George, J. M. 1992. Extrinsic and Intrinsic Origins of Perceived Social Loafing in Organizations. The Academy of Management Journal. 35, 1 (Mar. 1992), 191--202.

[5]

Granovetter, M. S. 2005. The Impact of Social Structure on Economic Outcomes. J. Econ. Persp. 19, 1 (2005), 33--50.

[6]

Hackman, J. R. and Oldham, G. R. 1976. Motivation through the design of work: test of a theory. Org. Behav. and Hum. Perf. 16, 2 (1976), 250--279.

[7]

Harris, M. 2008. Email from America. Sunday Times.

[8]

Huberman, B. A. et al. 2009. Crowdsourcing, attention and productivity. J. of Info. Sci. 35, 6 (Dec. 2009), 758--765.

Digital Library

[9]

Jans, L. et al. 2011. The induction of shared identity: The positive role of individual distinctiveness for groups. Pers. soc. psych. bulletin. 37, 8 (2011), 1130--1141.

[10]

Jones, G. R. 1984. Task Visibility, Free Riding, and Shirking: Explaining the Effect of Structure and Technology on Employee Behavior. Academy of Management Review. 9, 4 (Oct. 1984), 684--695.

[11]

Karau, S. J. and Williams, K. D. 1993. Social Loafing: A Meta-Analytic Review and Theoretical Integration. J. of Pers. and Soc.Psych. 65, 4 (1993), 681--706.

[12]

Kidwell, R. E. and Bennett, N. 1993. Employee propensity to withhold effort: A conceptual model to intersect three avenues of research. Acad. of Mgmt 1993.

[13]

Kittur, A. et al. 2008. Crowdsourcing user studies with Mechanical Turk. CHI 08. 08, April 5--10 (2008), 453.

Digital Library

[14]

Kittur, A. et al. 2011. CrowdForge: Crowdsourcing Complex Work. HCI (2011), 1801--1806.

Digital Library

[15]

Kulkarni, A. et al. 2012. Collaboratively crowdsourcing workflows with turkomatic. CSCW 2012.

Digital Library

[16]

Lakens, D. 2010. Movement synchrony and perceived entitativity. J. of Exp. Soc. Psych. 46, 5 (2010), 701--708.

[17]

Mason, W. and Watts, D. J. 2010. Financial incentives and the "performance of crowds." ACM SIGKDD Explorations Newsletter. 11, 2 (2010), 100.

Digital Library

[18]

Roland E. Kidwell, J. and Bennett, N. 1993. Employee Propensity to Withhold Effort: A Conceptual Model to Intersect Three Avenues of Research. The Academy of Management Review. 18, 3 (Jul. 1993), 429--456.

[19]

Shaw, A. D. et al. 2011. Designing Incentives for Inexpert Human Raters. CSCW. 2011.

Digital Library

[20]

Snow, R. et al. 2008. Cheap and Fast - But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. Emp. Meth. in Nat Lang. Proc. 2008.

Digital Library

[21]

Ware, C. 2000. Information visualization: perception for design. Morgan Kaufmann Publishers Inc.

Digital Library

Cited By

Bhatti SGavurova BAhmed AMarcone MSantoro G(2024)The impact of digital platforms on the creativity of remote workers through the mediating role of explicit and tacit knowledge sharingJournal of Knowledge Management10.1108/JKM-08-2023-068228:8(2433-2459)Online publication date: 6-May-2024
https://doi.org/10.1108/JKM-08-2023-0682
Sun YDing ZZhang ZGauthier J(2020)The Sustainable Positive Effects of Enterprise Social Media on Employees: The Visibility and Vicarious Learning LensSustainability10.3390/su1207285512:7(2855)Online publication date: 3-Apr-2020
https://doi.org/10.3390/su12072855
Li TManns CNorth CLuther K(2019)Dropping the Baton?Proceedings of the ACM on Human-Computer Interaction10.1145/33592383:CSCW(1-26)Online publication date: 7-Nov-2019
https://dl.acm.org/doi/10.1145/3359238
Show More Cited By

Index Terms

Workflow transparency in a microtask marketplace
1. Human-centered computing

Recommendations

Co-worker transparency in a microtask marketplace
CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative work

Workers in microtask work environments such as Mechanical Turk typically do not know if or how they fit into a workflow. The research question we posed here was whether displaying information about the number of other workers doing the same task would ...
The Grid Resource Broker workflow engine
2nd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2007)

Increasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and-or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread ...
Design and implementation of a workflow-based resource broker with information system on computational grids

The grid is a promising infrastructure that can allow scientists and engineers to access resources among geographically distributed environments. Grid computing is a new technology which focuses on aggregating resources (e.g., processor cycles, disk ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GROUP '12: Proceedings of the 2012 ACM International Conference on Supporting Group Work

October 2012

342 pages

ISBN:9781450314862

DOI:10.1145/2389176

General Chairs:
Thomas A. Finholt
University of Michigan, USA
,
Hilda Tellioğlu
Vienna University of Technology, Austria
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Tom Gross
University of Bamburg, Germany

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Group '12

Sponsor:

SIGCHI

Group '12: ACM 2012 International Conference on Support Group Work

October 27 - 31, 2012

Florida, Sanibel Island, USA

Acceptance Rates

Overall Acceptance Rate 125 of 405 submissions, 31%

Upcoming Conference

GROUP '25

Sponsor:
sigchi

The 2025 ACM International Conference on Supporting Group Work

January 12 - 15, 2025

Hilton Head , SC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
313
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bhatti SGavurova BAhmed AMarcone MSantoro G(2024)The impact of digital platforms on the creativity of remote workers through the mediating role of explicit and tacit knowledge sharingJournal of Knowledge Management10.1108/JKM-08-2023-068228:8(2433-2459)Online publication date: 6-May-2024
https://doi.org/10.1108/JKM-08-2023-0682
Sun YDing ZZhang ZGauthier J(2020)The Sustainable Positive Effects of Enterprise Social Media on Employees: The Visibility and Vicarious Learning LensSustainability10.3390/su1207285512:7(2855)Online publication date: 3-Apr-2020
https://doi.org/10.3390/su12072855
Li TManns CNorth CLuther K(2019)Dropping the Baton?Proceedings of the ACM on Human-Computer Interaction10.1145/33592383:CSCW(1-26)Online publication date: 7-Nov-2019
https://dl.acm.org/doi/10.1145/3359238
Mahyar NJames MNg MWu RDow SMandryk RHancock MPerry MCox A(2018)CommunityCritProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173769(1-14)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173769
Alkhatib ABernstein MLevi MMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Examining Crowd Work and Gig Work Through The Historical Lens of PieceworkProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025974(4599-4616)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025974
Newell ERuths DKaye JDruin ALampe CMorris DHourcade J(2016)How One Microtask Affects AnotherProceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858490(3155-3166)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2858036.2858490
Nguyen DDabbish LKiesler SCosley DForte ACiolfi LMcDonald D(2015)The Perverse Effects of Social Transparency on Online Advice TakingProceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing10.1145/2675133.2675253(207-217)Online publication date: 28-Feb-2015
https://dl.acm.org/doi/10.1145/2675133.2675253
Leonardi P(2014)Social Media, Knowledge Sharing, and InnovationInformation Systems Research10.1287/isre.2014.053625:4(796-816)Online publication date: 1-Dec-2014
https://dl.acm.org/doi/10.1287/isre.2014.0536
Marlow JDabbish LJones MPalanque PSchmidt AGrossman T(2014)Who's the boss?CHI '14 Extended Abstracts on Human Factors in Computing Systems10.1145/2559206.2581262(2533-2538)Online publication date: 26-Apr-2014
https://dl.acm.org/doi/10.1145/2559206.2581262

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents