research-article

Is Virtual Citizen Science A Game?

Authors:

Chris Phethean,

Ramine TinatiAuthors Info & Claims

ACM Transactions on Social Computing, Volume 1, Issue 2

Article No.: 6, Pages 1 - 39

https://doi.org/10.1145/3209960

Published: 27 June 2018 Publication History

Abstract

The use of game elements within virtual citizen science is increasingly common, promising to bring increased user activity, motivation, and engagement to large-scale scientific projects. However, there is an ongoing debate about whether or not gamifying systems such as these is actually an effective means by which to increase motivation and engagement in the long term. While gamification itself is receiving a large amount of attention, there has been little beyond individual studies to assess its suitability or success for citizen science; similarly, while frameworks exist for assessing citizen science performance, they tend to lack any appreciation of the effects that game elements might have had. We therefore review the literature to determine what the trends are regarding the performance of particular game elements or characteristics in citizen science, and survey existing projects to assess how popular different game features are. Investigating this phenomenon further, we then present the results of a series of interviews carried out with the EyeWire citizen science project team to understand more about how gamification elements are introduced, monitored, and assessed in a live project. Our findings suggest that projects use a range of game elements with points and leaderboards the most popular, particularly in projects that describe themselves as “games.” Currently, gamification appears to be effective in citizen science for maintaining engagement with existing communities, but shows limited impact for attracting new players.

References

[1]

David P. Anderson, Jeff Cobb, Eric Korpela, Matt Lebofsky, and Dan Werthimer. 2002. SETI@ home: An experiment in public-resource computing. Commun. ACM 45, 11 (2002), 56--61.

Digital Library

[2]

Andrés Francisco Aparicio, Francisco Luis Gutiérrez Vela, José Luis González Sánchez, and José Luis Isla Montes. 2012. Analysis and application of gamification. In Proceedings of the 13th International Conference on Interacción Persona-Ordenador (INTERACCION’12). ACM, New York, Article 17, 2 pages.

Digital Library

[3]

Avinoam Baruch, Andrew May, and Dapeng Yu. 2016. The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform. Computers in Human Behavior 64 (2016), 923--931.

[4]

Dr. Ivo Blohm and Prof. Dr. Jan M. Leimeister. 2013. Gamification. Business 8 Information Systems Engineering 5, 4 (Aug. 2013), 275--278. https://search.proquest.com/docview/1412812105?accountid=13963

[5]

John Bohannon. 2014. Online video game plugs players into remote-controlled biochemistry lab. Science 343, 6170 (2014), 475--475. arXiv:http://science.sciencemag.org/content/343/6170/475.full.pdf.

[6]

Anne Bowser, Derek Hansen, Yurong He, Carol Boston, Matthew Reid, Logan Gunnell, and Jennifer Preece. 2013. Using gamification to inspire new citizen science volunteers. In Proceedings of the 1st International Conference on Gameful Design, Research, and Applications. ACM, 18--25.

Digital Library

[7]

A. Carlier, A. Salvador, F. Cabezas, X. Giro-i Nieto, V. Charvillat, and O. Marques. 2016. Assessment of crowdsourcing and gamification loss in user-assisted object segmentation. Multimedia Tools and Applications 75, 23 (2016), 15901--15928.

Digital Library

[8]

Chen Chen, Paweł W. Woźniak, Andrzej Romanowski, Mohammad Obaid, Tomasz Jaworski, Jacek Kucharski, Krzysztof Grudzień, Shengdong Zhao, and Morten Fjeld. 2016. Using crowdsourcing for scientific analysis of industrial tomographic images. ACM Transactions on Intelligent Systems and Technology 7, 4 (July 2016), Article 52, 25 pages.

Digital Library

[9]

C. Chung, A. Kadan, Y. Yang, A. Matsuoka, J. Rubin, and M. Chechik. 2017. The impact of visual load on performance in a human-computation game. ACM International Conference Proceeding Series Part F130151.

Digital Library

[10]

Joe Cox, Eun Young Oh, Brooke Simmons, Chris Lintott, Karen Masters, Gary Graham, Anita Greenhill, and Kate Holmes. 2015. How is success defined and measured in online citizen science? A case study of Zooniverse projects.

[11]

Kevin Crowston and Isabelle Fagnot. 2008. The motivational arc of massive virtual collaboration. In Proceedings of the IFIP WG, Vol. 9.

[12]

Kevin Crowston and Nathan R. Prestopnik. 2013. Motivation and data quality in a citizen science game: A design science evaluation. In Proceedings of the 2013 46th Hawaii International Conference on System Sciences (HICSS). IEEE, 450--459.

Digital Library

[13]

Vickie Curtis. 2015. Motivation to participate in an online citizen science game: A study of Foldit. Science Communication 37, 6 (2015), 723--746.

[14]

Edward L. Deci, Richard Koestner, and Richard M. Ryan. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin 125, 6 (1999), 627--668.

[15]

Sebastian Deterding. 2011. Situated motivational affordances of game elements: A conceptual model. In Proceedings of Gamification: Using Game Design Elements in Non-gaming Contexts, A Workshop at CHI.

[16]

Sebastian Deterding. 2012. Gamification: Designing for motivation. Interactions 19, 4 (2012), 14--17.

Digital Library

[17]

Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011a. From game design elements to gamefulness: Defining gamification. In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. ACM, 9--15.

Digital Library

[18]

Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O’Hara, and Dan Dixon. 2011b. Gamification. using game-design elements in non-gaming contexts. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. ACM, 2425--2428.

Digital Library

[19]

Nicolas Ducheneaut, Robert J. Moore, and Eric Nickell. 2007. Virtual “third places”: A case study of sociability in massively multiplayer games. Proceedings of Computer Supported Cooperative Work (CSCW’07) 16, 1--2 (2007), 129--166.

Digital Library

[20]

David Easley and Arpita Ghosh. 2016. Incentives, gamification, and game theory: An economic approach to badge design. ACM Transactions on Economics and Computation 4, 3 (June 2016), Article 16, 26 pages.

Digital Library

[21]

Alexandra Eveleigh, Charlene Jennett, Ann Blandford, Philip Brohan, and Anna L. Cox. 2014. Designing for dabblers and deterring drop-outs in citizen science. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2985--2994.

Digital Library

[22]

Alexandra Eveleigh, Charlene Jennett, Stuart Lynn, and Anna L Cox. 2013. “I want to be a captain&excl; I want to be a captain&excl;”: Gamification in the old weather citizen science project. In Proceedings of the 1st International Conference on Gameful Design, Research, and Applications. ACM, 79--82.

Digital Library

[23]

Seamus F. Forde, Elisa D. Mekler, and Klaus Opwis. 2016. Informational, but not intrinsically motivating gamification: Preliminary findings. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (CHI PLAY Companion’16). ACM, New York, 157--163.

Digital Library

[24]

D. H.-L. Goh, R. P. Ang, A. Y. K. Chua, and C. S. Lee. 2010. Evaluating game genres for tagging images, In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI’10), 659--662.

Digital Library

[25]

G. G. Graham, J. Cox, Brooke Simmons, Chris Lintott, Karen Masters, A. Greenhill, and K. Holmes. 2015. How is success defined and measured in online citizen science: A case study of Zooniverse projects. Computing in Science and Engineering 99 (2015), 22.

[26]

A. Greenhill, K. Holmes, J. Woodcock, C. Lintott, B. D. Simmons, G. Graham, J. Cox, E. Y. Oh, and K. Masters. 2016. Playing with science: Exploring how game activity motivates users participation on an online citizen science platform. Aslib Journal of Information Management 68, 3 (2016), 306--325.

[27]

Juho Hamari. 2013. Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service. Electronic Commerce Research and Applications 12, 4 (2013), 236--245.

[28]

Juho Hamari, Jonna Koivisto, and Harri Sarsa. 2014. Does gamification work?—A literature review of empirical studies on gamification. In Proceedings of the 2014 47th Hawaii International Conference on System Sciences. IEEE, 3025--3034.

Digital Library

[29]

Jeff Howe. 2006. The rise of crowdsourcing. Wired Magazine 14, 6 (2006), 1--4.

[30]

Shih-Wen Huang and Wai-Tat Fu. 2012. Systematic analysis of output agreement games: Effects of gaming environment, social interaction, and feedback. Urbana 51 (2012), 61801.

[31]

Kai Huotari and Juho Hamari. 2012. Defining gamification: A service marketing perspective. In Proceedings of the 16th International Academic MindTrek Conference. ACM, 17--22.

Digital Library

[32]

Ioanna Iacovides and Anna L Cox. 2015. Moving beyond fun: Evaluating serious experience in digital games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2245--2254.

Digital Library

[33]

Ioanna Iacovides, Charlene Jennett, Cassandra Cornish-Trestrail, and Anna L. Cox. 2013. Do games attract or sustain engagement in citizen science?: A study of volunteer motivations. In Proceedings of CHI’13 Extended Abstracts on Human Factors in Computing Systems. ACM, 1101--1106.

Digital Library

[34]

Charlene Jennett, Laure Kloetzer, Daniel Schneider, Ioanna Iacovides, Anna Cox, Margaret Gold, Brian Fuchs, Alexandra Eveleigh, Kathleen Methieu, Zoya Ajani, and others. 2016. Motivations, learning and creativity in online citizen science. Journal of Science Communication 15, 3 (2016).

[35]

Ippokratis Kapenekakis and Konstantinos Chorianopoulos. 2017. Citizen science for pedestrian cartography: Collection and moderation of walkable routes in cities through mobile gamification. Human-centric Computing and Information Sciences 7, 1 (2017), 10.

Digital Library

[36]

Alexander Kawrykow, Gary Roumanis, Alfred Kam, Daniel Kwak, Clarence Leung, Chu Wu, Eleyine Zarour, Luis Sarmenta, Mathieu Blanchette, Jérôme Waldispühl, and others. 2012. Phylo: A citizen science approach for improving multiple sequence alignment. PloS One 7, 3 (2012), e31362.

[37]

Jeffrey Laut, Francesco Cappa, Oded Nov, and Maurizio Porfiri. 2016. Increasing citizen science contribution using a virtual peer. Journal of the Association for Information Science and Technology (2016).

[38]

Andreas Lieberoth. 2015. Shallow gamification testing psychological effects of framing an activity as a game. Games and Culture 10, 3 (2015), 229--248.

[39]

Chris J. Lintott, Kevin Schawinski, Anže Slosar, Kate Land, Steven Bamford, Daniel Thomas, M. Jordan Raddick, Robert C. Nichol, Alex Szalay, Dan Andreescu, and others. 2008. Galaxy Zoo: Morphologies derived from visual inspection of galaxies from the Sloan digital sky survey. Monthly Notices of the Royal Astronomical Society 389, 3 (2008), 1179--1189.

[40]

Jane McGonigal. 2011. Reality is Broken: Why Games Make us Better And How They Can Change the World. Penguin.

Digital Library

[41]

Elisa D. Mekler, Florian Brühlmann, Klaus Opwis, and Alexandre N. Tuch. 2013. Disassembling gamification: The effects of points and meaning on user motivation and performance. In Proceedings of CHI’13 Extended Abstracts on Human Factors in Computing Systems. ACM, 1137--1142.

Digital Library

[42]

Benedikt Morschheuser, Juho Hamari, and Jonna Koivisto. 2016. Gamification in crowdsourcing: A review. In Proceedings of the 2016 49th Hawaii International Conference on System Sciences (HICSS’16). IEEE, 4375--4384.

Digital Library

[43]

Benedikt Morschheuser, Juho Hamari, Jonna Koivisto, and Alexander Maedche. 2017. Gamified crowdsourcing: Conceptualization, literature review, and future agenda. International Journal of Human-Computer Studies 106 (2017), 26--43.

[44]

Oded Nov, Ofer Arazy, and David Anderson. 2011. Dusting for science: Motivation and participation of digital citizen science volunteers. In Proceedings of the 2011 iConference. ACM, 68--74.

Digital Library

[45]

M. K. Pedersen, N. R. Rasmussen, J. Sherson, and R. V. Basaiawmoit. 2017. Leaderboard effects on player performance in a citizen science game. Proceedings of the 11th European Conference on Games Based Learning (ECGBL’17), 531--537.

[46]

M. Ponti, T. Hillman, and I. Stankovic. 2015. Science and gamification: The odd couple? In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY’15), 679--684.

Digital Library

[47]

Catherine Pope, Sue Ziebland, Nicholas Mays, and others. 2000. Analysing qualitative data. BMJ 320, 7227 (2000), 114--116.

[48]

C. Preist, E. Massung, and D. Coyle. 2014. Competing or aiming to be average? Normification as a means of engaging digital volunteers. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW’14), 1222--1233.

Digital Library

[49]

Nathan Prestopnik, Kevin Crowston, and Jun Wang. 2014. Exploring data quality in games with a purpose. In Proceedings of iConference 2014.

[50]

N. Prestopnik, K. Crowston, and J. Wang. 2017. Gamers, citizen scientists, and data: Exploring participant contributions in two games with a purpose. Computers in Human Behavior 68 (Mar. 2017), 254--268.

Digital Library

[51]

Nathan R. Prestopnik and Jian Tang. 2015. Points, stories, worlds, and diegesis: Comparing player experiences in two citizen science games. Computers in Human Behavior 52 (2015), 492--506.

Digital Library

[52]

M. Jordan Raddick, Georgia Bracey, Pamela L. Gay, Chris J. Lintott, Carie Cardamone, Phil Murray, Kevin Schawinski, Alexander S. Szalay, and Jan Vandenberg. 2013. Galaxy zoo: Motivations of citizen scientists. Astronomy Education Review 12, 1 (2013).

[53]

Neal Reeves, Ramine Tinati, Sergej Zerr, Max G. Van Kleek, and Elena Simperl. 2017. From crowd to community: A survey of online community features in citizen science projects. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 2137--2152.

Digital Library

[54]

Francesco Restuccia, Sajal K. Das, and Jamie Payton. 2016. Incentive mechanisms for participatory sensing: Survey and research challenges. ACM Transactions on Sensor Networks 12, 2 (April 2016), Article 13, 40 pages.

Digital Library

[55]

Markus Rokicki, Sergej Zerr, and Stefan Siersdorfer. 2016. Just in time: Controlling temporal performance in crowdsourcing competitions. In Proceedings of the 25th International Conference on World Wide Web (WWW’16). International World Wide Web Conferences Steering Committee, 817--827.

Digital Library

[56]

Dana Rotman, Jen Hammock, Jenny Preece, Derek Hansen, Carol Boston, Anne Bowser, and Yurong He. 2014. Motivations affecting initial and long-term participation in citizen science projects in three countries. In Proceedings of iConference 2014.

[57]

Dana Rotman, Jenny Preece, Jen Hammock, Kezee Procita, Derek Hansen, Cynthia Parr, Darcy Lewis, and David Jacobs. 2012. Dynamic changes in motivation in collaborative citizen-science projects. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW’12). ACM, New York, 217--226.

Digital Library

[58]

David Rudrum. 2005. From narrative representation to narrative use: Towards the limits of definition. Narrative 13, 2 (2005), 195--204.

[59]

M. Sabou, K. Bontcheva, A. Scharl, and M. Föls. 2013. Games with a purpose or mechanised labour? A comparative study, In Proceedings of the 13th International Conference on Knowledge Management and Knowledge Technologies. ACM International Conference Proceeding Series, (2013), Article 19, 8 pages.

Digital Library

[60]

Katie Seaborn and Deborah I. Fels. 2015. Gamification in theory and action: A survey. International Journal of Human-Computer Studies 74 (2015), 14--31.

Digital Library

[61]

K. Siu, M. Guzdial, and M. O. Riedl. 2017. Evaluating singleplayer and multiplayer in human computation games. ACM International Conference Proceedings Series Part F130151.

Digital Library

[62]

K. Siu and M. O. Riedl. 2016. Reward systems in human computation games, In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY’16), 266--275.

Digital Library

[63]

Kristin Siu, Alexander Zook, and Mark O. Riedl. 2014. Collaboration versus competition: Design and evaluation of mechanics for games with a purpose. In Proceedings of the 9th International Conference on the Foundations of Digital Games.

[64]

S. Thaler, E. Simperl, and S. Wölger. 2012. An experiment in comparing human-computation techniques. IEEE Internet Computing 16, 5 (sep-oct 2012), 52--58.

Digital Library

[65]

Sarah-Kristin Thiel. 2016. A review of introducing game elements to e-participation. In Proceedings of the Conference for E-Democracy and Open Government (CeDEM’16). IEEE, 3--9.

[66]

Sarah-Kristin Thiel, Michaela Reisinger, Kathrin Röderer, and Peter Fröhlich. 2016. Playing (with) democracy: A review of gamified participation approaches. JeDEM-eJournal of eDemocracy and Open Government 8, 3 (2016), 32--60.

[67]

Jennifer Thom, David Millen, and Joan DiMicco. 2012. Removing gamification from an enterprise SNS. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work. ACM, 1067--1070.

Digital Library

[68]

Ramine Tinati, Markus Luczak-Roesch, Elena Simperl, and Wendy Hall. 2016. Because science is awesome: Studying participation in a citizen science game. In Proceedings of the 8th ACM Conference on Web Science. ACM, 45--54.

Digital Library

[69]

R. Tinati, M. Luczak-Roesch, E. Simperl, and W. Hall. 2017a. An investigation of player motivations in EyeWire, a gamified citizen science project. Computers in Human Behavior 73 (Aug. 2017), 527--540.

Digital Library

[70]

Ramine Tinati, Elena Simperl, and Markus Luczak-Roesch. 2017b. To help or hinder: Real-time chat in citizen science. In Proceedings of the11th International Conference on Web and Social Media. https://eprints.soton.ac.uk/406181/

[71]

Ramine Tinati, Max Van Kleek, Elena Simperl, Markus Luczak-Rösch, Robert Simpson, and Nigel Shadbolt. 2015. Designing for citizen data analysis: A cross-sectional case study of a multi-domain citizen science platform. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, 4069--4078.

Digital Library

[72]

A. Wiggins and K. Crowston. 2011. From conservation to crowdsourcing: A typology of citizen science. In Proceedings of the 2011 44th Hawaii International Conference on System Sciences. 1--10.

Digital Library

Cited By

Maddalena EIbáñez LReeves NSimperl E(2023)Qrowdsmith: Enhancing Paid Microtask Crowdsourcing with Gamification and Furtherance IncentivesACM Transactions on Intelligent Systems and Technology10.1145/360494014:5(1-26)Online publication date: 21-Jun-2023
https://dl.acm.org/doi/10.1145/3604940
Cicognani SStein STonin MVlassopoulos M(2023)Symbolic incentives and the recruitment of volunteers for citizen science projectsOxford Economic Papers10.1093/oep/gpad03175:4(923-940)Online publication date: 3-Aug-2023
https://doi.org/10.1093/oep/gpad031
Khalili-Mahani NHolowka EWoods SKhaled RRoy MLashley MGlatard TTimm-Bottos JDahan ANiesters MHovey RSimon BKirmayer L(2021)Play the Pain: A Digital Strategy for Play-Oriented Research and ActionFrontiers in Psychiatry10.3389/fpsyt.2021.74647712Online publication date: 15-Dec-2021
https://doi.org/10.3389/fpsyt.2021.746477
Show More Cited By

Index Terms

Is Virtual Citizen Science A Game?
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Computer supported cooperative work

Recommendations

Do games attract or sustain engagement in citizen science?: a study of volunteer motivations
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Increasingly, games are being incorporated into online citizen science (CS) projects as a way of crowdsourcing data; yet the influence of gamification on volunteer motivations and engagement in CS projects is still unknown. In an interview study with 8 ...
“I want to be a captain! I want to be a captain!”: gamification in the old weather citizen science project
Gamification '13: Proceedings of the First International Conference on Gameful Design, Research, and Applications

Gamification is increasingly implemented in citizen science projects as a means of motivating and sustaining participation. In a survey and subsequent interviews we explored the appeal of gamification for participants in the Old Weather project, and its ...
Gamifying citizen science: a study of two user groups
CSCW Companion '14: Proceedings of the companion publication of the 17th ACM conference on Computer supported cooperative work & social computing

Citizen science projects increasingly incorporate the motivational affordances of games. However, the different user groups that gamified citizen science projects may attract are poorly understood. This project examines how two user groups, nature ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Social Computing

ACM Transactions on Social Computing Volume 1, Issue 2

June 2018

102 pages

EISSN:2469-7826

DOI:10.1145/3234932

Editor:
Kevin Crowston
Syracuse University, USA

Issue’s Table of Contents

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2018

Accepted: 01 April 2018

Revised: 01 April 2018

Received: 01 June 2017

Published in TSC Volume 1, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Theory and Practice of Social Machines
Stars4All EUHorizon 2020 Framework Programme
Digital Economy programme
Engineering and Physical Sciences Research Council on behalf of UK Research and Innovation
UK Engineering and Physical Sciences Research Council (EPSRC) SOCIAM

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
527
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Maddalena EIbáñez LReeves NSimperl E(2023)Qrowdsmith: Enhancing Paid Microtask Crowdsourcing with Gamification and Furtherance IncentivesACM Transactions on Intelligent Systems and Technology10.1145/360494014:5(1-26)Online publication date: 21-Jun-2023
https://dl.acm.org/doi/10.1145/3604940
Cicognani SStein STonin MVlassopoulos M(2023)Symbolic incentives and the recruitment of volunteers for citizen science projectsOxford Economic Papers10.1093/oep/gpad03175:4(923-940)Online publication date: 3-Aug-2023
https://doi.org/10.1093/oep/gpad031
Khalili-Mahani NHolowka EWoods SKhaled RRoy MLashley MGlatard TTimm-Bottos JDahan ANiesters MHovey RSimon BKirmayer L(2021)Play the Pain: A Digital Strategy for Play-Oriented Research and ActionFrontiers in Psychiatry10.3389/fpsyt.2021.74647712Online publication date: 15-Dec-2021
https://doi.org/10.3389/fpsyt.2021.746477
Diethei DNiess JStellmacher CStefanidi ESchöning JKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Sharing Heartbeats: Motivations of Citizen Scientists in Times of CrisesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445665(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445665
Loria ENacke LPirker J(2021)The Quirks of Being a Wallflower: Towards Defining Lurkers and Loners in Games Through A Systematic Literature ReviewExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3451830(1-7)Online publication date: 8-May-2021
https://dl.acm.org/doi/10.1145/3411763.3451830
Robinson JKocman DSpeyer OGerasopoulos E(2021)Meeting volunteer expectations — a review of volunteer motivations in citizen science and best practices for their retention through implementation of functional features in CS toolsJournal of Environmental Planning and Management10.1080/09640568.2020.1853507(1-31)Online publication date: 2-Feb-2021
https://doi.org/10.1080/09640568.2020.1853507
Gutiérrez Páez NGómez-Cañón JPorcaro LSantos PHernández-Leo DGómez E(2021)Emotion Annotation of Music: A Citizen Science ApproachCollaboration Technologies and Social Computing10.1007/978-3-030-85071-5_4(51-66)Online publication date: 25-Aug-2021
https://doi.org/10.1007/978-3-030-85071-5_4
Cedazo RGonzalez ESerra-Ricart MBrunete A(2020)Improving the Results of Citizen Science Projects Through Reputation Systems: The Case of Wolf’s Number ExperimentIEEE Access10.1109/ACCESS.2020.30300068(186026-186038)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3030006
Budde MRohe JHirschoff LSchlosser PBeigl MHolopainen JSchankin A(2020)SpaceMaze: incentivizing correct mobile crowdsourced sensing behaviour with a sensified minigameBehaviour & Information Technology10.1080/0144929X.2020.177287840:15(1627-1642)Online publication date: 4-Jun-2020
https://doi.org/10.1080/0144929X.2020.1772878
Ibáñez LReeves NSimperl E(2020) Crowdsourcing and Human‐in‐the‐Loop for IoT The Internet of Things10.1002/9781119545293.ch8(91-105)Online publication date: 6-Mar-2020
https://doi.org/10.1002/9781119545293.ch8
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents