research-article

Private Peer Feedback as Engagement Driver in Humanitarian Mapping

Authors:

Licia CapraAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 1, Issue CSCW

Article No.: 40, Pages 1 - 18

https://doi.org/10.1145/3134675

Published: 06 December 2017 Publication History

Abstract

Prior research suggests that public negative feedback on social knowledge sharing platforms can be powerfully demotivating to newcomers, particularly when it involves peer feedback mechanisms such as ratings and commenting systems. What is the impact on newcomer retention when feedback is private, and from a single peer reviewer? We study these effects using the example of the Humanitarian OpenStreetMap Team, a Wikipedia-style social mapping platform where the review process is closer to a teacher-learner model rather than a public peer review. We observe peer feedback for early contributions by 1,300 newcomers, and assess the impact of different classes of feedback, including performance feedback, corrective feedback, and verbal rewards. We find that verbal rewards and immediate feedback can have a powerful effect on newcomer retention. In order to better support such positive engagement effects, we recommend that system designers conceptually distinguish between mechanisms for quality control and for learner feedback.

References

[1]

Panagiotis Adamopoulos. 2013. What makes a great MOOC? An interdisciplinary analysis of student retention in online courses ICIS.

[2]

June Ahn, Brian S. Butler, Cindy Weng, and Sarah Webster. 2013. Learning to be a better Q'er in social Q&A sites: social norms and information artifacts. Proceedings of the American Society for Information Science and Technology, Vol. 50, 1 (2013), 1--10.

Digital Library

[3]

Blake K Ashforth and Alan M Saks. 1996. Socialization tactics: Longitudinal effects on newcomer adjustment. Academy of management Journal Vol. 39, 1 (1996), 149--178.

[4]

Jonathan Bishop. 2007. Increasing participation in online communities: A framework for human-computer interaction. Computers in human behavior Vol. 23, 4 (2007), 1881--1893.

Digital Library

[5]

Joel Brockner, William R Derr, and Wesley N Laing. 1987. Self-esteem and reactions to negative feedback: Toward greater generalizability. Journal of Research in Personality Vol. 21, 3 (1987), 318--333.

[6]

Judy Cameron, Katherine M Banko, and W David Pierce. 2001. Pervasive negative effects of rewards on intrinsic motivation: The myth continues. The Behavior Analyst, Vol. 24, 1 (2001), 1.

[7]

Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How Community Feedback Shapes User Behavior. In Proc. ICWSM '14.

[8]

Boreum Choi, Kira Alexander, Robert E Kraut, and John M Levine. 2010. Socialization tactics in Wikipedia and their effects Proc. CSCW '10. ACM, 107--116.

Digital Library

[9]

Edward L Deci, Richard Koestner, and Richard M Ryan. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin Vol. 125, 6 (1999), 627.

[10]

Chrysanthos Dellarocas. 2003. The digitization of word of mouth: Promise and challenges of online feedback mechanisms. Management science, Vol. 49, 10 (2003), 1407--1424.

Digital Library

[11]

Martin Dittus, Giovanni Quattrone, and Licia Capra. 2016. Analysing volunteer engagement in humanitarian mapping: building contributor communities at large scale. In Proc. CSCW '16.

Digital Library

[12]

Martin Dittus, Giovanni Quattrone, and Licia Capra. 2017. Mass participation during emergency response: event-centric crowd-sourcing in humanitarian mapping. In Proc. CSCW '17.

Digital Library

[13]

Melanie Eckle and Jo ao Porto de Albuquerque. 2015. Quality assessment of remote mapping in OpenStreetMap for disaster management purposes Proc. ISCRAM '15.

[14]

Laurence S Freedman. 1982. Tables of the number of patients required in clinical trials using the logrank test. Statistics in medicine Vol. 1, 2 (1982), 121--129.

[15]

Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work Proc. WikiSym '11. ACM, 163--172.

Digital Library

[16]

Judith M Harackiewicz. 1979. The effects of reward contingency and performance feedback on intrinsic motivation. Journal of personality and social psychology, Vol. 37, 8 (1979), 1352.

[17]

John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research Vol. 77, 1 (2007), 81--112.

[18]

Aniket Kittur, Bryan Pendleton, and Robert E Kraut. 2009. Herding the cats: the influence of groups in coordinating peer production Proc. WikiSym '09. ACM, 7.

Digital Library

[19]

Robert E Kraut, Paul Resnick, Sara Kiesler, Moira Burke, Yan Chen, Niki Kittur, Joseph Konstan, Yuqing Ren, and John Riedl. 2012. Building successful online communities: Evidence-based social design. Mit Press.

Digital Library

[20]

J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159--174.

[21]

Jingyan Lu and Nancy Law. 2012. Online peer assessment: effects of cognitive and affective feedback. Instructional Science Vol. 40, 2 (2012), 257--275.

[22]

Richard S Lysakowski and Herbert J Walberg. 1982. Instructional effects of cues, participation, and corrective feedback: A quantitative synthesis. American Educational Research Journal Vol. 19, 4 (1982), 559--572.

[23]

Loizos Michael and Jahna Otterbacher. 2014. Write Like I Write: Herding in the Language of Online Reviews. Proc. ICWSM '14.

[24]

Richard L Moreland and Paul D Sweeney. 1984. Self-expectancies and reactions to evaluations of personal performance. Journal of Personality Vol. 52, 2 (1984), 156--176.

[25]

Jonathan T Morgan, Siko Bouterse, Heather Walls, and Sarah Stierch. 2013. Tea and sympathy: crafting positive new user experiences on wikipedia Proc. CSCW '13. ACM, 839--848.

Digital Library

[26]

David R Musicant, Yuqing Ren, James A Johnson, and John Riedl. 2011. Mentoring in Wikipedia: a clash of cultures. In Proc. WikiSym '11. ACM, 173--182.

Digital Library

[27]

Leysia Palen, Robert Soden, T Jennings Anderson, and Mario Barrenechea. 2015. Success & scale in a data-producing organization: the socio-technical evolution of OpenStreetMap in response to humanitarian events Proc. SIGCHI '15. 4113--4122.

Digital Library

[28]

Katherine Panciera, Aaron Halfaker, and Loren Terveen. 2009. Wikipedians are born, not made: a study of power editors on Wikipedia Proc. GROUP '09. ACM, 51--60.

Digital Library

[29]

Laurence D Parnell, Pierre Lindenbaum, Khader Shameer, Giovanni Marco Dall'Olio, Daniel C Swan, Lars Juhl Jensen, Simon J Cockell, Brent S Pedersen, Mary E Mangan, Christopher A Miller, et almbox. 2011. BioStar: an online question & answer resource for the bioinformatics community. PLoS Comput Biol, Vol. 7, 10 (2011), e1002216.

[30]

Paul Resnick, Ko Kuwabara, Richard Zeckhauser, and Eric Friedman. 2000. Reputation systems. Commun. ACM Vol. 43, 12 (2000), 45--48.

Digital Library

[31]

Robert Soden and Leysia Palen. 2014. From crowdsourced mapping to community mapping: the post-earthquake work of OpenStreetMap Haiti. (2014).

[32]

Yla R Tausczik and James W Pennebaker. 2012. Participation in an online mathematics community: differentiating motivations to add Proc. CSCW '12. ACM, 207--216.

Digital Library

[33]

Gershon Tenenbaum and Ellen Goldring. 1989. A meta-analysis of the effect of enhanced instruction: Cues, participation, reinforcement and feedback and correctives on motor skill learning. Journal of Research & Development in Education (1989).

[34]

Clay Westrope, Robert Banick, and Mitch Levine. 2014. Groundtruthing OpenStreetMap building damage assessment. Procedia engineering Vol. 78 (2014), 29--39.

[35]

Donghee Yvette Wohn. 2015. The Effects of Feedback and Habit on Content Posting in an Online Community. iConference 2015 Proceedings (2015).

[36]

Haiyi Zhu, Amy Zhang, Jiping He, Robert E Kraut, and Aniket Kittur. 2013. Effects of peer feedback on contribution: a field experiment in Wikipedia Proc. SIGCHI '13. ACM, 2253--2262.

Digital Library

Cited By

Warncke-Wang MHo RMiller MJohnson I(2023)Increasing Participation in Peer Production Communities with the Newcomer HomepageProceedings of the ACM on Human-Computer Interaction10.1145/36100717:CSCW2(1-26)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610071
Choe YTomko MKalantari M(2023)Assessing Mapper Conflict in OpenStreetMap Using the Delphi Survey MethodProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580758(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580758
Maitland CWong-Villacres MPereira RBahamóndez EPeres LBaguma R(2022)Digital Platform Use for Refugee Crisis ResponseProceedings of the 2022 International Conference on Information and Communication Technologies and Development10.1145/3572334.3572376(1-5)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3572334.3572376
Show More Cited By

Index Terms

Private Peer Feedback as Engagement Driver in Humanitarian Mapping
1. Human-centered computing
  1. Collaborative and social computing

Recommendations

Large-scale Volunteer Engagement in Humanitarian Mapping
CSCW '16 Companion: Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion

Organisers of large crowdsourcing initiatives need to con-sider how to produce outcomes with their projects, but also how to build volunteer capacity. The initial contributor ex-perience plays an important role in this, particularly when contributions ...
Effects of peer feedback on contribution: a field experiment in Wikipedia
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

One of the most significant challenges for many online communities is increasing members' contributions over time. Prior studies on peer feedback in online communities have suggested its impact on contribution, but have been limited by their ...
Analysing Volunteer Engagement in Humanitarian Mapping: Building Contributor Communities at Large Scale
CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

Organisers of large-scale crowdsourcing initiatives need to consider how to produce outcomes with their projects, but also how to build volunteer capacity. The initial project experience of contributors plays an important role in this, particularly when ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 1, Issue CSCW

November 2017

2095 pages

EISSN:2573-0142

DOI:10.1145/3171581

Editors:
Clifford Lampe
University of Michigan
,
Jeff Nichols
Google
,
Karrie Karahalios
University of Illinois Urbana-Champaign
,
Geraldine Fitzpatrick
Vienna University of Technology
,
Uichin Lee
KAIST
,
Andres Monroy-Hernandez
Snap Inc.
,
Wolfgang Stuerzlinger
Simon Fraser University

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2017

Published in PACMHCI Volume 1, Issue CSCW

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
175
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)8

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Warncke-Wang MHo RMiller MJohnson I(2023)Increasing Participation in Peer Production Communities with the Newcomer HomepageProceedings of the ACM on Human-Computer Interaction10.1145/36100717:CSCW2(1-26)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610071
Choe YTomko MKalantari M(2023)Assessing Mapper Conflict in OpenStreetMap Using the Delphi Survey MethodProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580758(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580758
Maitland CWong-Villacres MPereira RBahamóndez EPeres LBaguma R(2022)Digital Platform Use for Refugee Crisis ResponseProceedings of the 2022 International Conference on Information and Communication Technologies and Development10.1145/3572334.3572376(1-5)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3572334.3572376
Khanal KBudhathoki NErbstein N(2019)Filling OpenStreetMap data gaps in rural Nepal: a digital youth internship and leadership ProgrammeOpen Geospatial Data, Software and Standards10.1186/s40965-019-0071-14:1Online publication date: 21-Oct-2019
https://doi.org/10.1186/s40965-019-0071-1
Chiang CKasunic ASavage S(2018)Crowd CoachProceedings of the ACM on Human-Computer Interaction10.1145/32743062:CSCW(1-17)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1145/3274306

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents