research-article

Open access

Crowdsourcing as a Tool for Research: Implications of Uncertainty

Authors:

Krzysztof Z. Gajos,

Andrea Wiggins,

Alex WilliamsAuthors Info & Claims

CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

Pages 1544 - 1561

https://doi.org/10.1145/2998181.2998197

Published: 25 February 2017 Publication History

Abstract

Numerous crowdsourcing platforms are now available to support research as well as commercial goals. However, crowdsourcing is not yet widely adopted by researchers for generating, processing or analyzing research data. This study develops a deeper understanding of the circumstances under which crowdsourcing is a useful, feasible or desirable tool for research, as well as the factors that may influence researchers' decisions around adopting crowdsourcing technology. We conducted semi-structured interviews with 18 researchers in diverse disciplines, spanning the humanities and sciences, to illuminate how research norms and practitioners' dispositions were related to uncertainties around research processes, data, knowledge, delegation and quality. The paper concludes with a discussion of the design implications for future crowdsourcing systems to support research.

References

[1]

Adam J. Berinsky, Gregory A. Huber, and Gabriel S. Lenz. 2012. Evaluating online labor markets for experimental research: Amazon.com's mechanical turk. Political Analysis 20, 3 (2012), 351--368.

[2]

Jeremy P. Birnholtz and Matthew J. Bietz. 2003. Data at Work: Supporting Sharing in Science and Engineering. In Proceedings of the International ACM SIGGROUP Conference on Supporting Group Work (GROUP '03). ACM, New York, NY, USA, 339--348.

Digital Library

[3]

David Blumenthal, Eric G Campbell, Manjusha Gokhale, Recai Yucel, Brian Clarridge, Stephen Hilgartner, and Neil a Holtzman. 2006. Data withholding in genetics and the other life sciences: prevalences and predictors. Academic Medicine 81, 2 (2006), 137--145.

[4]

Sara Bly. 1998. Special Section on Collaboratories. interactions 5, 3 (May 1998), 31.

Digital Library

[5]

John Bohannon. 2016. Psychologists grow increasingly dependent on online research subjects. (2016). Retrieved June 7, 2016 from http://www.sciencemag.org/news/2016/06/psychologistsgrow-increasingly-dependent-online-research-subjects.

[6]

Rick Bonney, Caren B. Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, Kenneth V. Rosenberg, and Jennifer Shirk. 2009. Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience 59, 11 (2009), 977--984.

[7]

Anne Bowser, Derek Hansen, Jennifer Preece, Yurong He, Carol Boston, and Jen Hammock. 2014. Gamifying Citizen Science: A Study of Two User Groups. In Proceedings of the Companion Publication of the 17th ACM Conference on Computer Supported Cooperative Work (CSCW Companion '14). ACM, New York, NY, USA, 137--140.

Digital Library

[8]

Anne Bowser-Livermore and Andrea Wiggins. 2015. Privacy in Participatory Research: Advancing Policy to support Human Computation. Human Computation 2, 1 (2015), 19--44.

[9]

Alex Bowyer, Chris Lintott, Greg Hines, Campbell Allan, and Ed Paget. 2015. Panoptes, a Project Building Tool for Citizen Science. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP '15). AAAI, San Diego, CA, USA, 1--2.

[10]

H.K. Burgess, L.B. DeBey, H.E. Froehlich, N. Schmidt, E.J. Theobald, A.K. Ettinger, J. HilleRisLambers, J. Tewksbury, and J.K. Parrish. 2016. The science of citizen science: Exploring barriers to use as a primary research tool. Biological Conservation (2016), 1--8.

[11]

Carolin Cardamone, Kevin Schawinski, Marc Sarzi, Steven P. Bamford, Nicola Bennert, C. M. Urry, Chris Lintott, William C. Keel, John Parejko, Robert C. Nichol, Daniel Thomas, Dan Andreescu, Phil Murray, M. Jordan Raddick, Anze Slosar, Alex Szalay, and Jan Vandenberg. 2009. Galaxy Zoo Green Peas: Discovery of a class of compact extremely star-forming galaxies. Monthly Notices of the Royal Astronomical Society 399, 3 (2009), 1191--1205.

[12]

Casey Trees. 2016. Citizen Science. (2016). Retrieved January 15, 2016 from http://caseytrees.org/get-involved/citizen-science/.

[13]

Karin Knorr Cetina. 1999. Epistemic Communities: How the Sciences Make Knowledge. Harvard Education Press, Cambridge, MA.

[14]

Sayan Chakraborti, Naveen Yadav, Carolin Cardamone, and Alak Ray. 2012. Radio Detection of Green Peas: Implications for Magnetic Fields in Young Galaxies. The Astrophysical Journal 746, 1 (2012), L6.

[15]

Jeffrey P Cohn. 2008. Citizen Science : Can Volunteers Do Real Research? BioScience 58, 3 (2008), 192--197.

[16]

Harry Collins. 1992. Changing Order: Replication and Induction in Scientific Practice. University of Chicago Press, Chicago, IL.

[17]

Harry Collins and Robert Evans. 2007. Rethinking Expertise. University of Chicago Press.

[18]

Colorado State University. 2016. Citsci.Org. (2016). Retrieved January 15, 2016 from http://www.citsci.org.

[19]

Caren Cooper. 2016. Zen in the Art of Citizen Science: Apps for Collective Discovery and the 1 Percent Rule of the Web. (2016). Retrieved January 15, 2016 from http://blogs.scientificamerican.com/guest-blog/zen-inthe-art-of-citizen-science-apps-for-collectivediscovery-and-the-1-percent-rule-of-the-web/.

[20]

Juliet M. Corbin and Anselm Strauss. 2008. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (3rd ed.). Sage, Thousand Oaks, CA.

[21]

M. L. Cornwell and L. M. Campbell. 2012. Co-producing conservation and knowledge: Citizen-based sea turtle monitoring in North Carolina, USA. Social Studies of Science 42, 1 (2012), 101--120.

[22]

Joe Cox, Eun Young Oh, Brooke Simmons, Chris Lintott, Karen Masters, Anita Greenhill, Gary Graham, and Kate Holmes. 2015. Defining and Measuring Success in Online Citizen Science: A Case Study of Zooniverse Projects. IEEE Computing in Science & Engineering 17, 4 (2015), 28--41.

Digital Library

[23]

M. H. Cragin, C. L. Palmer, J. R. Carlson, and M. Witt. 2010. Data sharing, small science and institutional repositories. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 368, 1926 (2010), 4023--4038.

[24]

Schlagwein Daniel and Daneshgar Farhad. 2014. User requirements of a crowdsourcing platform for researchers : findings from a series of focus groups. In Proceedings of the 2014 Pacific Asia Conference on Information Systems. 1--11.

[25]

Janet Dickinson and Rick Bonney. 2012. Citizen Science : Public Participation in Environmental Research. Cornstock Pub. Associates, Ithaca, NY.

[26]

Michael Dummett. 1991. The Logical Basis of Metaphysics. Duckworth, London, UK.

[27]

Kevin Dunbar. 2000. How Scientists Think in the Real World. Journal of Applied Developmental Psychology 21, 1 (2000), 49--58.

[28]

Alexandra Eveleigh, Charlene Jennett, Ann Blandford, Philip Brohan, and Anna L. Cox. 2014. Designing for Dabblers and Deterring Drop-outs in Citizen Science. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2985--2994.

Digital Library

[29]

Elizabeth J Farnsworth, Miyoko Chu, W John Kress, Amanda K Neill, Jason H Best, John Pickering, Robert D Stevenson, Gregory W Courtney, John K V Dyk, and Aaron M Ellison. 2013. Next-Generation Field Guides. BioScience 63, 11 (2013), 891--899.

[30]

Karen Fort, Gilles Adda, and K. Bretonnel Cohen. 2011. Amazon Mechanical Turk: Gold Mine or Coal Mine? Computational Linguistics 37, 2 (June 2011), 413--420.

Digital Library

[31]

Amy Freitag and Max Pfeffer. 2013. Process, Not Product: Investigating Recommendations for Improving Citizen Science Success. PLoS One 8, 5 (2013), e64079.

[32]

Bruno S. Frey and Reto Jegen. 2001. Motivation Crowding Theory. Journal of Economic Surveys 15, 5 (2001), 589--611.

[33]

Jay R Galbraith. 1974. Organization Design: An Information Processing View. Interfaces 4, 3 (1974), 28--36.

Digital Library

[34]

Thomas F. Giergyn. 1999. Cultural Boundaries of Science: Credibility on the Line. University of Chicago Press, Chicago, IL.

[35]

Thomas F Gieryn. 1983. Boundary-Work and the Demarcation of Science From Non-Science: Strains and Interests in Professional Ideologies of Scientists. American Sociological Review 48, 6 (1983), 781--795.

[36]

Elizabeth Goodman and Janet Vertesi. 2012. Design for X?: Distribution Choices and Ethical Design. In Extended Abstracts of the 30th Annual ACM Conference on Human Factors in Computing Systems (CHI EA '12). ACM, New York, NY, USA, 81--90.

Digital Library

[37]

David Grier. 2007. When Computers Were Humans. Princeton University Press, Princeton, NJ.

Digital Library

[38]

Katie Hafner and Matthew Lyon. 1996. Where Wizards Stay up Late: the Origins of the Internet. Simon & Schuster, New York.

Digital Library

[39]

Muki Haklay. 2013. Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. Springer, Berlin, Chapter Citizen Science and Volunteer Geographic Information: Overview and Typology of Participation, 105--122.

[40]

Harvard School of Engineering and Applied Sciences. 2016. Privacy Tools for Sharing Research Data. (2016). Retrieved January 15, 2016 from http://privacytools.seas.harvard.edu.

[41]

Imperial College London. 2016. EpiCollect.net Website. (2016). Retrieved January 15, 2016 from http://www.epicollect.net.

[42]

Yuri I. Izotov, Natalia G. Guseva, and Trinh X. Thuan. 2011. Green Pea Galaxies and Cohorts: Luminous Compact Emission-Line Galaxies in the Sloan Digital Sky Survey. The Astrophysical Journal 728, 2 (2011), 161.

[43]

Corey Jackson, Gabriel Mugar, Kevin Crowston, and Carsten Osterlund. 2016. Encouraging Work in Citizen Science: Experiments in Goal Setting and Anchoring. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW '16 Companion). ACM, New York, NY, USA, 297--300.

Digital Library

[44]

A. E. Jaskot and M.S. Oey. 2013. The Origin and Optical Depth of Ionizing Radiation in the "Green Pea" Galaxies. The Astrophysical Journal 766 (2013), 91.

[45]

Marina Jirotka, Charlotte P. Lee, and Gary M. Olson. 2013. Supporting scientific collaboration: Methods, tools and concepts. Computer Supported Cooperative Work: CSCW: An International Journal 22, 4--6 (2013), 667--715.

Digital Library

[46]

Sunyoung Kim, Jennifer Mankoff, and Eric Paulos. 2013. Sensr: Evaluating a Flexible Framework for Authoring Mobile Data-collection Tools for Citizen Science. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (CSCW '13). ACM, New York, NY, USA, 1453--1462.

Digital Library

[47]

Thomas Kuhn. 2012. The Structure of Scientific Revolutions (4th ed.). University of Chicago Press, Chicago, IL.

[48]

Deepak Kulkarni and Herbert A. Simon. 1988. The processes of scientific discovery: The strategy of experimentation. Cognitive Science 12, 2 (1988), 139--175.

[49]

Bruno Latour and Steve Woolgar. 1986. Laboratory life: the Social Construction of Scientific Facts (2nd ed.). Princeton University Press, Beverly Hills, CA.

[50]

Edith Law, Connor Dalton, Nick Merrill, Alex Young, and Krzysztof Z. Gajos. 2013. Curio: a platform for supporting mixed-expertise crowdsourcing. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP '13). AAAI, 1--2.

[51]

Chris J. Lintott, Kevin Schawinski, Anze Slosar, Kate Land, Steven Bamford, Daniel Thomas, M. Jordan Raddick, Robert C. Nichol, Alex Szalay, Dan Andreescu, Phil Murray, and Jan Vandenberg. 2008. Galaxy Zoo: Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society 389, 3 (2008), 1179--1189.

[52]

Christopher S. Lowry and Michael N. Fienen. 2013. CrowdHydrology: Crowdsourcing hydrologic data and engaging citizen scientists. GroundWater 51, 1 (2013), 151--156.

[53]

Marder, Jenny and Fritz, Mike. (????).

[54]

Winter Mason and Siddharth Suri. 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior Research Methods 44, 1 (2012), 1--23.

[55]

R. Merton, E. Barber, and J. Shulman. 2006. The Travels and Adventures of Serendipity: A Study in Sociological Semantics and the Sociology of Science. Princeton University Press.

[56]

New York University. 2016. Project Noah. (2016). Retrieved January 15, 2016 from http://www.projectnoah.org.

[57]

Jon Noronha, Eric Hysen, Haoqi Zhang, and Krzysztof Z. Gajos. 2011. Platemate: Crowdsourcing Nutritional Analysis from Food Photographs. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 1--12.

Digital Library

[58]

Gary Olson, Ann Zimmerman, Nathan Bos, and William Wulf. 2008. Scientific Collaboration on the Internet. MIT Press, Cambridge, MA.

[59]

Gary M. Olson, Daniel E. Atkins, Robert Clauer, Thomas A. Finholt, Farnam Jahanian, Timothy L. Killeen, Atul Prakash, and Terry Weymouth. 1998. The Upper Atmospheric Research Collaboratory (UARC). interactions 5, 3 (May 1998), 48--55.

Digital Library

[60]

Johan Oomen and Lora Aroyo. 2011. Crowdsourcing in the Cultural Heritage Domain : Opportunities and Challenges. C&T '11 Proceedings of the 5th International Conference on Communities and Technologies July (2011), 138--149.

Digital Library

[61]

Michael Polanyi. 1967. The Tacit Dimension. Routledge and Kegan Paul, London, UK.

[62]

J. Preece and B. Schneiderman. 2009. The Reader-to-Leader Framework: Motivating Technology-Mediated Social Participation. AIS Transactions on Human-Computer Interaction 1, 1 (2009), 13--32.

[63]

Nancy Proctor. 2013. Crowdsourcing--An Introduction: From Public Goods to Public Good. Curator: The Museum Journal 56, 1 (January 2013), 105--106.

[64]

Danial Qaurooni, Ali Ghazinejad, Inna Kouper, and Hamid Ekbia. 2016. Citizens for Science and Science for Citizens: The View from Participatory Design. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1822--1826.

Digital Library

[65]

M. Jordan Raddick, Georgia Bracey, Pamela L. Gay, Chris J. Lintott, Phil Murray, Kevin Schawinski, Alexander S. Szalay, and Jan Vandenberg. 2010. Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. Astronomy Education Review 9, 1 (2010), 010103.

[66]

Katharina Reinecke and Krzysztof Z. Gajos. 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work (CSCW '15). ACM, New York, NY, USA, 1364--1378.

Digital Library

[67]

Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Walter S. Lasecki, Jay Patel, Negar Rahmati, Tulsee Doshi, Melissa Valentine, and Michael S. Bernstein. 2014. Expert Crowdsourcing with Flash Teams. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 75--85.

Digital Library

[68]

Hauke Riesch and Clive Potter. 2014. Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions. Public Understanding of Science 23, 1 (2014), 107--120.

[69]

Dana Rotman, Jenny Preece, Jen Hammock, Kezee Procita, Derek Hansen, Cynthia Parr, Darcy Lewis, and David Jacobs. 2012. Dynamic Changes in Motivation in Collaborative Citizen-science Projects. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW '12). ACM, New York, NY, USA, 217--226.

Digital Library

[70]

Sharman Apt Russell. 2014. Diary of a Citizen Scientist. Oregon State University Press, Corvallis, OR.

[71]

SciFabric. 2016a. CrowdCrafting.org Website. (2016). Retrieved January 15, 2016 from http://crowdcrafting.org.

[72]

SciFabric. 2016b. Pybossa. (2016). Retrieved January 15, 2016 from http://www.pybossa.com.

[73]

SciStarter. 2016. Project Finder. (2016). Retrieved January 15, 2016 from http://scistarter.com/finder.

[74]

S. Shapin. 1995. Here and Everywhere: Sociology of Scientific Knowledge. Annual Review of Sociology 21, 1 (1995), 289--321.

[75]

Jennifer Shirk. 2014. Push the Edge of Science Forward: Expanding Considerations of Expertise Through Scientists' Citizen Science Work in Conservation. Ph.D. Dissertation. Cornell University, Ithaca, NY.

[76]

Jennifer L. Shirk, Heidi L. Ballard, Candie C. Wilderman, Tina Phillips, Andrea Wiggins, Rebecca Jordan, Ellen McCallie, Matthew Minarchek, Bruce V. Lewenstein, Marianne E. Krasny, and Rick Bonney. 2012. Public participation in scientific research: A framework for deliberate design. Ecology and Society 17, 2 (2012).

[77]

Jonathan Silvertown. 2009. A new dawn for citizen science. Trends in ecology & evolution 24, 9 (2009), 467--71.

[78]

Robert Simpson, Kevin R. Page, and David De Roure. 2014. Zooniverse: Observing the World's Largest Citizen Science Platform. In Proceedings of the 23rd International Conference on World Wide Web (WWW '14 Companion). ACM, New York, NY, USA, 1049--1054.

Digital Library

[79]

Ryo Suzuki, Niloufar Salehi, Michelle S. Lam, Juan C. Marroquin, and Michael S. Bernstein. 2016. Atelier: Repurposing Expert Crowdsourcing Tasks As Micro-internships. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2645--2656.

Digital Library

[80]

Carol Tenopir, Suzie Allard, Kimberly Douglass, Arsev Umur Aydinoglu, Lei Wu, Eleanor Read, Maribeth Manoff, and Mike Frame. 2011. Data sharing by scientists: Practices and perceptions. PLoS ONE 6, 6 (2011).

[81]

Ramine Tinati, Max Van Kleek, Elena Simperl, Markus Luczak-Rösch, Robert Simpson, and Nigel Shadbolt. 2015. Designing for Citizen Data Analysis: A Cross-Sectional Case Study of a Multi-Domain Citizen Science Platform. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 4069--4078.

Digital Library

[82]

Sharon Traweek. 1988. Beamtimes and Lifetimes: the World of High Energy Physicists. Harvard University Press, Cambridge, MA.

[83]

Michael L Tushman and David A Nadler. 1978. Information Processing as an Integrating Concept in Organizational Design. Source: The Academy of Management Review 3, 3 (1978), 613--624.

[84]

Amos Tversky and Daniel Kahneman. 1974. Judgment under uncertainty: heuristics and biases. Science 185, 4157 (1974), 1124--1131.

[85]

University College of London. 2016. Transcribe Bentham: A Participatory Initiative. (2016). Retrieved January 15, 2016 from http://blogs.ucl.ac.uk/transcribe-bentham/.

[86]

Andrew J Westphal, Anna L Butterworth, Christopher J Snead, Nahide Craig, David Anderson, Steven M Jones, Donald E Brownlee, Richard Farnsworth, and Michael E Zolensky. 2005. Stardust@home: A Massively Distributed Public Search for Interstellar Dust in the Stardust Interstellar Dust Collector. In Lunar and Planetary Science XXXVI. http://ntrs.nasa.gov/search.jsp?R=20050180792

[87]

Andrea Wiggins. 2012. Crowdsourcing Scientific Work: A Comparative Study of Technologies, Processes, and Outcomes in Citizen Science. Ph.D. Dissertation. Syracuse University, Syracuse, NY.

[88]

Andrea Wiggins and Kevin Crowston. 2011. From Conservation to Crowdsourcing: A Typology of Citizen Science. In Proceedings of the 44th Hawaii International Conference on System Sciences (HICSS '11). IEEE Computer Society, Washington, DC, USA, 1--10.

Digital Library

[89]

Andrea Wiggins and Kevin Crowston. 2012. Goals and Tasks: Two Typologies of Citizen Science Projects. In Proceedings of the 44th Hawaii International Conference on System Sciences (HICSS '12). IEEE Computer Society, Washington, DC, USA, 3426--3435.

Digital Library

[90]

A. Wiggins, G. Newman, R.D. Stevenson, and K. Crowston. 2011. Mechanisms for Data Quality and Validation in Citizen Science. In IEEE seventh International Conference on e-Science Workshops (eScienceW '11). 14--19.

Digital Library

[91]

Alex C Williams, John F Wallin, Haoyu Yu, Marco Perale, Hyrum D Carroll, Anne-Francoise Lamblin, Lucy Fortson, Dirk Obbink, Chris J Lintott, and James H Brusuelas. 2014. A computational pipeline for crowdsourced transcriptions of Ancient Greek papyrus fragments. In Proceedings of 2014 IEEE International Conference on Big Data. 100--105.

[92]

Chris Wood, Brian Sullivan, Marshall Iliff, Daniel Fink, and Steve Kelling. 2011. eBird: Engaging birders in science and conservation. PLoS Biology 9, 12 (2011).

[93]

A Zimmerman. 2008. New knowledge from old data The role of standards in the sharing and reuse of ecological data. Science Technology & Human Values 33, 5 (2008), 631--652.

[94]

Ann Zimmerman, Bos Nathan, Judy Olson, and Gary Olson. 2009. e-Research: Transformation in Scholarly Practice. Routledge, Chapter The Promise of Data in e-Research: Many Challenges, Multiple Solutions, Diverse Outcomes, 222--239.

[95]

Zooniverse. 2016a. AnnoTate. (2016). Retrieved January 15, 2016 from https://anno.tate.org.uk/.

[96]

Zooniverse. 2016b. Old Weather Project. (2016). Retrieved January 15, 2016 from http://www.oldweather.org/.

[97]

Zooniverse. 2016c. Operation War Diary. (2016). Retrieved January 15, 2016 from http://www.operationwardiary.org/.

Cited By

Bonicalza SValsecchi ECoppola ECapatano VThatcher H(2024)Citizen science in eDNA monitoring for mediterranean monk seal conservationBMC Ecology and Evolution10.1186/s12862-024-02338-824:1Online publication date: 24-Dec-2024
https://doi.org/10.1186/s12862-024-02338-8
Lenart-Gansiniec R(2024)Management scholars’ perception of barriers and bridges to use crowdsourcing in science: qualitative researchStudies in Higher Education10.1080/03075079.2024.231242449:12(2546-2559)Online publication date: 20-Feb-2024
https://doi.org/10.1080/03075079.2024.2312424
Jiang YBenatallah BBáez M(2024)Understanding how early-stage researchers leverage socio-technical affordances for distributed research supportInformation and Software Technology10.1016/j.infsof.2023.107340165:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107340
Show More Cited By

Index Terms

Crowdsourcing as a Tool for Research: Implications of Uncertainty
1. Human-centered computing

Recommendations

Older Adults’ Motivation and Engagement with Diverse Crowdsourcing Citizen Science Tasks
Human-Computer Interaction – INTERACT 2021
Abstract
In this exploratory study we evaluated the engagement, performance and preferences of older adults who interacted with different citizen science tasks. Out of 40 projects recently active on the Zooniverse platform we selected top ones to be ...
Crowdsourcing for agricultural applications

We reviewed crowdsourcing activities in agriculture and classified them in 4 categories.We identified 8 types of inputs that can be collected by crowdsourcing for agricultural applications.We discussed data quality and contributors participation.We ...
Understanding geocaching practices and motivations
CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Geocaching is a location-based activity that has been practiced for a number of years. As a sustained and established activity it represents an important opportunity for understanding everyday practices and motivations that can build up around a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

February 2017

2556 pages

ISBN:9781450343350

DOI:10.1145/2998181

General Chairs:
Charlotte P. Lee
University of Washington
,
Steve Poltrock
Retired
,
Program Chairs:
Louise Barkhuus
Stockholm University and Cornell Tech
,
Marcos Borges
Universidade Federal do Rio de Janeiro
,
Wendy Kellogg
IBM T.J. Watson Research Center

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSERC Discovery Grant

Conference

CSCW '17

Sponsor:

SIGCHI

CSCW '17: Computer Supported Cooperative Work and Social Computing

February 25 - March 1, 2017

Oregon, Portland, USA

Acceptance Rates

CSCW '17 Paper Acceptance Rate 183 of 530 submissions, 35%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

46
Total Citations
View Citations
2,039
Total Downloads

Downloads (Last 12 months)446
Downloads (Last 6 weeks)57

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bonicalza SValsecchi ECoppola ECapatano VThatcher H(2024)Citizen science in eDNA monitoring for mediterranean monk seal conservationBMC Ecology and Evolution10.1186/s12862-024-02338-824:1Online publication date: 24-Dec-2024
https://doi.org/10.1186/s12862-024-02338-8
Lenart-Gansiniec R(2024)Management scholars’ perception of barriers and bridges to use crowdsourcing in science: qualitative researchStudies in Higher Education10.1080/03075079.2024.231242449:12(2546-2559)Online publication date: 20-Feb-2024
https://doi.org/10.1080/03075079.2024.2312424
Jiang YBenatallah BBáez M(2024)Understanding how early-stage researchers leverage socio-technical affordances for distributed research supportInformation and Software Technology10.1016/j.infsof.2023.107340165:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107340
Lenart-Gansiniec RCzakon WMeyer N(2024)Antecedents of researchers’ behavioral intentions to use crowdsourcing in science: a multilevel approachReview of Managerial Science10.1007/s11846-024-00797-3Online publication date: 15-Aug-2024
https://doi.org/10.1007/s11846-024-00797-3
Paulino DCorreia AYagui MBarroso JLiberato MVivacqua AGrover ABigham JParedes H(2023)Exploring Stigmergic Collaboration and Task Modularity Through an Expert Crowdsourcing Annotation System: The Case of Storm Phenomena in the Euro-Atlantic RegionIEEE Access10.1109/ACCESS.2023.331959711(106485-106502)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3319597
Askratni OMostefai S(2022)Crowdsourced Document Summarization as a Support for Research PublicationInternational Journal of Organizational and Collective Intelligence10.4018/IJOCI.31359812:1(1-24)Online publication date: 4-Nov-2022
https://dl.acm.org/doi/10.4018/IJOCI.313598
Soteropoulos DMarsico T(2022)Community Science Success for Herbarium Transcription in Arkansas: Building a Network of Students and Volunteers for Notes from NatureCastanea10.2179/0008-7475.87.1.5487:1Online publication date: 1-Jun-2022
https://doi.org/10.2179/0008-7475.87.1.54
Yurman PReddy AMaiden NSas C(2022)Drawing Conversations Mediated by AIProceedings of the 14th Conference on Creativity and Cognition10.1145/3527927.3531448(56-70)Online publication date: 20-Jun-2022
https://dl.acm.org/doi/10.1145/3527927.3531448
Wani ASingh DSingh P(2022)“Hartal (Strike) Happens Here Everyday”: Understanding Impact of Disruption on Education in KashmirProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502126(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502126
Hall MAfzali MKrause MCaton S(2022)What Quality Control Mechanisms do We Need for High-Quality Crowd Work?IEEE Access10.1109/ACCESS.2022.320729210(99709-99723)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3207292
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents