Challenges in Restructuring Community-based Moderation
Authors: 
Chau Tran, Kejsi Take, Kaylea Champion, 
Benjamin Mako Hill, Rachel GreenstadtAuthors Info & Claims
Chau Tran, Kejsi Take, Kaylea Champion, Benjamin Mako Hill, Rachel GreenstadtAuthors Info & ClaimsArticle No.: 415, Pages 1 - 24
Abstract
Content moderation practices and technologies need to change over time as requirements and community expectations shift. However, attempts to restructure the existing moderation practices can be difficult, especially for platforms that rely on their communities to moderate, because changes can transform the workflow and workload participants' reward systems. By examining the extensive archival discussions around a prepublication moderation technology on Wikipedia named Flagged Revisions, complemented by seven semi-structured interviews, we identify various challenges in restructuring community-based moderation practices. Thus, we find that while a new system might sound good in theory and perform well in terms of quantitative metrics, it may conflict with existing social norms. Furthermore, our findings underscore how the relationship between platforms and self-governed communities can hinder the ability to assess the performance of any new system and introduce considerable costs related to maintaining, overhauling, or scrapping any piece of infrastructure.
References
[1]
2008. [Wikide-l] FlaggedRevs in der deutschsprachigen Wikipedia. https://lists.wikimedia.org/pipermail/wikidel/2008-June/020734.html.
[2]
2008. Wikipedia:Meinungsbilder/Weiterführung der gesichteten Versionen. https://de.wikipedia.org/wiki/Wikipedia:Meinungsbilder/Weiterf%C3%BChrung_der_gesichteten_Versionen.
[3]
2009. Wikipedia:Flagged revisions petition. https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions_petition.
[4]
2012. Wikipedia:Pending changes/Request for Comment 2012. https://en.wikipedia.org/wiki/Wikipedia:Pending_changes/Request_for_Comment_2012.
[5]
2017. FlaggedRevs for Norwegian (bokmål) Wikipedia. https://phabricator.wikimedia.org/T66726.
[6]
2019. Szerkeszt?:Tgr/Jelölt változatok hatásának elemzése/en. https://hu.wikipedia.org/wiki/Szerkeszt%C5%91:Tgr/Jel%C3%B6lt_v%C3%A1ltozatok_hat%C3%A1s%C3%A1nak_elemz%C3%A9se/en.
[7]
2020. Reconfigure FlaggedRevs at the Indonesian Wikipedia. https://phabricator.wikimedia.org/T268317.
[8]
2023. Community Wishlist Survey 2023. https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2023.
[9]
2023. Hilfe:Gesichtete Versionen. https://de.wikipedia.org/wiki/Hilfe:Gesichtete_Versionen.
[10]
2023. Moderator Code of Conduct. https://www.redditinc.com/policies/moderator-code-of-conduct.
[11]
2023. Wikipedia:Automated moderation. https://en.wikipedia.org/wiki/Wikipedia:Automated_moderation.
[12]
2023. Wikipedia:Protection policy. https://en.wikipedia.org/wiki/Wikipedia:Protection_policy.
[13]
2023. Wikipedia:Welcome unregistered editing. https://en.wikipedia.org/wiki/Wikipedia:Welcome_unregistered_editing.
[14]
2024. Flagged Revisions. https://meta.wikimedia.org/wiki/Flagged_Revisions.
[15]
2024. Page review statistics. https://pl.wikipedia.org/wiki/Specjalna:Statystyki_oznaczania.
[16]
2024. Statistiken zu gesichteten Versionen. https://de.wikipedia.org/wiki/Spezial:Sichtungsstatistik.
[17]
2024. Wikimedia Deutschland. https://meta.wikimedia.org/wiki/Wikimedia_Deutschland.
[18]
2024. Wikipedia:Bot policy. https://en.wikipedia.org/wiki/Wikipedia:Bot_policy.
[19]
2024. Wikipedia:Flagged protection and patrolled revisions. https://en.wikipedia.org/wiki/Wikipedia:Flagged_protection_and_patrolled_revisions.
[20]
Thomas J. Allen. 1984. Managing the flow of technology: Technology transfer and the dissemination of technological information within the R&D organization. MIT Press Books Volume 1 (1984).
[21]
Michael Barrett, Eivor Oborn, Wanda J Orlikowski, and JoAnne Yates. 2012. Reconfiguring boundary relations: Robotic innovations in pharmacy work. Organization Science 23, 5 (2012), 1448--1466.
[22]
Yochai Benkler. 2002. Coase's penguin, or, Linux and the nature of the firm. Yale Law Journal (2002), 369--446.
[23]
Yochai Benkler, Aaron Shaw, and Benjamin Mako Hill. 2015. Peer production: A form of collective intelligence. Handbook of collective intelligence 175 (2015).
[24]
Susan L Bryant, Andrea Forte, and Amy Bruckman. 2005. Becoming Wikipedian: Transformation of participation in a collaborative online encyclopedia. In Proceedings of the 2005 ACM International Conference on Supporting Group Work. 1--10.
[25]
Robyn Caplan, Lauren Hanson, and Joan Donovan. 2018. Dead reckoning: Navigating content moderation after 'fake news'. (2018).
[26]
Kaylea Champion, Nora McDonald, Stephanie Bankes, Joseph Zhang, Rachel Greenstadt, Andrea Forte, and Benjamin Mako Hill. 2019. A forensic qualitative analysis of contributions to Wikipedia from anonymity seeking users. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1--26.
[27]
Kathy Charmaz. 2014. Constructing Grounded Theory. SAGE.
[28]
Pedro Coutinho and Rui José. 2017. Moderation techniques for user-generated content in place-based communication. In 2017 12th Iberian Conference on Information Systems and Technologies (CISTI). 1--6. https://doi.org/10.23919/CISTI.2017.7975786
[29]
Allan Dafoe. 2015. On technological determinism: A typology, scope conditions, and a mechanism. Science, Technology, & Human Values 40, 6 (2015), 1047--1076.
[30]
Paul B de Laat. 2012. Coercion or empowerment? Moderation of content in Wikipedia as ?essentially contested' bureaucratic rules. Ethics and Information Technology 14 (2012), 123--135.
[31]
Michael A. DeVito, Ashley Marie Walker, and Jeremy Birnholtz. 2018. 'Too gay for Facebook': Presenting LGBTQ identity throughout the personal social media ecosystem. Proceedings of the ACM on Human-Computer Interaction 2, CSCW, Article 44 (November 2018), 23 pages. https://doi.org/10.1145/3274313
[32]
Julia R. Fernandez and Jeremy Birnholtz. 2019. "I don't want them to not know": Investigating decisions to disclose transgender identity on dating platforms. Proceedings of the ACM on Human-Computer Interaction 3, CSCW, Article 226 (November 2019), 21 pages. https://doi.org/10.1145/3359328
[33]
Andrea Forte, Vanesa Larco, and Amy Bruckman. 2009. Decentralization in Wikipedia governance. Journal of Management Information Systems 26, 1 (2009), 49--72.
[34]
Jiaojiao Fu, Yangfan Zhou, and Xin Wang. 2022. Unveiling high-speed follow-up consultation for chronic disease treatment: A pediatric hospital case in China. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2, Article 347 (November 2022), 29 pages. https://doi.org/10.1145/3568491
[35]
R. Stuart Geiger. 2014. Bots, bespoke code, and the materiality of software platforms. Information, Communication & Society 17, 3 (2014), 342--356. https://doi.org/10.1080/1369118X.2013.873069
[36]
R. Stuart Geiger and David Ribes. 2010. The Work of Sustaining Order in Wikipedia: The Banning of a Vandal. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (Savannah, Georgia, USA) (CSCW '10). Association for Computing Machinery, New York, NY, USA, 117--126. https://doi.org/10.1145/1718918.1718941
[37]
Tarleton Gillespie. 2018. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press, New Haven, Connecticut.
[38]
Tarleton Gillespie. 2020. Content moderation, AI, and the question of scale. Big Data & Society Volume 7 (2020).
[39]
James Grimmelmann. 2015. The virtues of moderation. Yale Journal of Law & Technology 17 (2015), 42.
[40]
Aaron Halfaker and R. Stuart Geiger. 2020. ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia. Proc. ACM Hum.-Comput. Interact. 4, CSCW2, Article 148 (oct 2020), 37 pages. https://doi.org/10.1145/3415219
[41]
Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, and John Riedl. 2013. The rise and decline of an open collaboration system: How Wikipedia's reaction to popularity is causing its decline. American Behavioral Scientist 57, 5 (2013), 664--688.
[42]
Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't bite the newbies: How reverts affect the quantity and quality of Wikipedia work. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (Mountain View, California) (WikiSym '11). Association for Computing Machinery, New York, NY, USA, 163--172. https://doi.org/10.1145/2038558.2038585
[43]
Benjamin Mako Hill and Aaron Shaw. 2021. The hidden costs of requiring accounts: Quasi-experimental evidence from peer production. Communication Research 48, 6 (2021), 771--795.
[44]
Shagun Jhaver, Iris Birman, Eric Gilbert, and Amy Bruckman. 2019. Human-Machine Collaboration for Content Regulation: The Case of Reddit Automoderator. ACM Transactions on Computer-Human Interaction 26, 5, Article 31 (July 2019), 35 pages. https://doi.org/10.1145/3338243
[45]
Shagun Jhaver, Iris Birman, Eric Gilbert, and Amy Bruckman. 2019. Human-machine collaboration for content regulation: The case of Reddit automoderator. ACM Transactions on Computer-Human Interaction 26, 5, Article 31 (jul 2019), 35 pages. https://doi.org/10.1145/3338243
[46]
Shagun Jhaver, Seth Frey, and Amy X Zhang. 2023. Decentralizing platform power: A design space of multi-level governance in online social platforms. Social Media Society 9, 4 (2023).
[47]
Jialun Aaron Jiang, Peipei Nie, Jed R. Brubaker, and Casey Fiesler. 2023. A trade-off-centered framework of content moderation. ACM Transactions on Computer-Human Interaction 30, 1, Article 3 (March 2023), 34 pages. https: //doi.org/10.1145/3534929
[48]
Bente Kalsnes and Karoline Andrea Ihlebæk. 2021. Hiding hate speech: Political moderation on Facebook. Media, Culture & Society 43, 2 (2021), 326--342.
[49]
Brian C. Keegan. 2012. Breaking news on Wikipedia: dynamics, structures, and roles in high-tempo collaboration. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work Companion. 315--318.
[50]
Charles Kiene, Kate Grandprey-Shores, Eshwar Chandrasekharan, Shagun Jhaver, Jialun "Aaron" Jiang, Brianna Dym, Joseph Seering, Sarah Gilbert, Kat Lo, Donghee Yvette Wohn, and Bryan Dosono. 2019. Volunteer Work: Mapping the Future of Moderation Research. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing (Austin, TX, USA) (CSCW '19). Association for Computing Machinery, New York, NY, USA, 492--497. https://doi.org/10.1145/3311957.3359443
[51]
Charles Kiene and Benjamin Mako Hill. 2020. Who uses bots' A statistical analysis of bot usage in moderation teams. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8. https://doi.org/10.1145/3334480.3382960
[52]
Charles Kiene, Jialun Aaron Jiang, and Benjamin Mako Hill. 2019. Technological Frames and User Innovation: Exploring Technological Change in Community Moderation Teams. Proceedings of the ACM on Human-Computer Interaction 3, CSCW, Article 44 (November 2019), 23 pages. https://doi.org/10.1145/3359146
[53]
Piotr Konieczny. 2009. Governance, organization, and democracy on the Internet: The iron law and the evolution of Wikipedia. In Sociological Forum, Vol. 24. Wiley Online Library, 162--192.
[54]
Robert E Kraut and Paul Resnick. 2012. Building successful online communities: Evidence-based social design. MIT Press.
[55]
Cliff Lampe and Paul Resnick. 2004. Slash (dot) and burn: distributed moderation in a large online conversation space. In Proceedings of the SIGCHI conference on Human factors in computing systems. 543--550.
[56]
Dorothy A. Leonard. 2011. Implementation as mutual adaptation of technology and organization. Managing Knowledge Assets, Creativity and Innovation 17 (2011), 429.
[57]
J Nathan Matias. 2019. The civic labor of volunteer moderators online. Social Media Society 5, 2 (2019).
[58]
Yiftach Nagar. 2012. What do you think? The structuring of an online community as a collective-sensemaking process. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (Seattle,Washington, USA) (CSCW '12). Association for Computing Machinery, New York, NY, USA, 393--402. https://doi.org/10.1145/2145204.2145266
[59]
Martin Oliver. 2011. Technological determinism in educational technology research: some alternative ways of thinking about the relationship between learning and technology. Journal of Computer Assisted Learning 27, 5 (2011), 373--384.
[60]
Wanda J. Orlikowski. 1992. Learning from Notes: Organizational issues in groupware implementation. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work (Toronto, Ontario, Canada) (CSCW '92). Association for Computing Machinery, New York, NY, USA, 362--369. https://doi.org/10.1145/143457.143549
[61]
Wanda J. Orlikowski. 1996. Improvising organizational transformation over time: A situated change perspective. Information Systems Research 7, 1 (1996), 63--92.
[62]
Wanda J. Orlikowski and Debra C. Gash. 1991. Changing frames: Understanding technological change in organizations. (1991).
[63]
Wanda J. Orlikowski and Debra C. Gash. 1994. Technological frames: Making sense of information technology in organizations. ACM Transactions on Information Systems (TOIS) 12, 2 (1994), 174--207.
[64]
Elinor Ostrom. 1990. Governing the commons: The evolution of institutions for collective action. Cambridge University Press.
[65]
Nathaniel Poor. 2005. Mechanisms of an online public sphere: The website Slashdot. Journal of Computer-Mediated Communication 10, 2 (2005), JCMC1028.
[66]
Joseph Seering. 2020. Reconsidering self-moderation: The role of research in supporting community-based models for online content moderation. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2, Article 107 (October 2020), 28 pages. https://doi.org/10.1145/3415178
[67]
Joseph Seering, TonyWang, Jina Yoon, and Geoff Kaufman. 2019. Moderator engagement and community development in the age of algorithms. New Media & Society 21, 7 (2019), 1417--1443.
[68]
Bongwon Suh, Gregorio Convertino, Ed H Chi, and Peter Pirolli. 2009. The singularity is not near: slowing growth of Wikipedia. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration. 1--10.
[69]
Nathan TeBlunthuis, Aaron Shaw, and Benjamin Mako Hill. 2018. Revisiting "The rise and decline" in a population of peer production projects. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3173574.3173929
[70]
Hibby Thach, Samuel Mayworm, Daniel Delmonaco, and Oliver Haimson. 2022. (In) visible moderation: A digital ethnography of marginalized users and content moderation on Twitch and Reddit. New Media & Society (2022).
[71]
Chau Tran, Kaylea Champion, Benjamin Mako Hill, and Rachel Greenstadt. 2022. The risks, benefits, and consequences of prepublication moderation: Evidence from 17 Wikipedia language editions. Proceedings of the ACM on Human- Computer Interaction 6, CSCW2, Article 333 (November 2022), 25 pages. https://doi.org/10.1145/3555225
[72]
Marlon Twyman, Brian C. Keegan, and Aaron Shaw. 2017. Black Lives Matter in Wikipedia: Collective Memory and Collaboration around Online Social Movements. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1400--1412. https://doi.org/10.1145/2998181.2998232
[73]
Lyudmila Vaseva and Claudia Müller-Birn. 2020. You shall not publish: Edit filters on English Wikipedia. In Proceedings of the 16th International Symposium on Open Collaboration (Virtual conference, Spain) (OpenSym '20). Association for Computing Machinery, New York, NY, USA, Article 4, 10 pages. https://doi.org/10.1145/3412569.3412580
Index Terms
- Challenges in Restructuring Community-based Moderation
Recommendations
The Risks, Benefits, and Consequences of Prepublication Moderation: Evidence from 17 Wikipedia Language Editions
CSCWMany online communities rely on postpublication moderation where contributors-even those that are perceived as being risky-are allowed to publish material immediately and where moderation takes place after the fact. An alternative arrangement involves ...
Practicing Moderation: Community Moderation as Reflective Practice
CSCW1Many types of online communities rely on volunteer moderators to manage the community and maintain behavioral standards. While prior work has shown that community moderators often develop a deep understanding of the goals of their moderation context and ...
How the R community creates and curates knowledge: an extended study of stack overflow and mailing lists
One of the effects of social media's prevalence in software development is the many flourishing communities of practice where users share a common interest. These large communities use many different communication channels, but little is known about how ...
Comments
Information & Contributors
Information
Published In
November 2024
5177 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 November 2024
Published in PACMHCI Volume 8, Issue CSCW2
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 76Total Downloads
- Downloads (Last 12 months)76
- Downloads (Last 6 weeks)37
Reflects downloads up to 26 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in