research-article

Public Access

Flag and Flaggability in Automated Moderation: The Case of Reporting Toxic Behavior in an Online Game Community

Authors:

Xinning GuiAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 437, Pages 1 - 12

https://doi.org/10.1145/3411764.3445279

Published: 07 May 2021 Publication History

All formats PDF

Abstract

Online platforms rely upon users or automated tools to flag toxic behaviors, the very first step in online moderation. While much recent research has examined online moderation, the role of flag remains poorly understood. This question becomes even more urgent in automated moderation, where flagging becomes a primary source of human judgment. We conducted a qualitative study of flagging practices in League of Legends (LoL), a popular eSports game. We found stark differences between how flag is designed to identify toxicity, and flaggability, or how players use and appropriate flag. Players distrust flag, but also appropriate flag for instrumental purposes. Thus, flaggability diverges decidedly from the conception of toxicity, and must be understood within the highly competitive gaming context of LoL. These findings help shed light on the situated nature of flaggability, the role of flag in online moderation, as well as implications for designing flag and moderation.

References

[1]

Leigh Achterbosch, Charlynn Miller, and Peter Vamplew. 2017. A taxonomy of griefer type by motivation in massively multiplayer online role-playing games. Behav. Inf. Technol. 36, 8 (August 2017), 846–860.

Digital Library

[2]

Sonam Adinolf and Selen Turkay. 2018. Toxic Behaviors in Esports Games: Player Perceptions and Coping Strategies. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts - CHI PLAY ’18 Extended Abstracts, 365–372.

Digital Library

[3]

Ali Alkhatib and Michael Bernstein. 2019. Street–level algorithms: A theory at the gaps between policy and decisions. In Conference on Human Factors in Computing Systems - Proceedings, 1–13.

Digital Library

[4]

Richard A. Bartle. 1996. Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDs. J. Virtual Environ. 1, 1 (1996).

[5]

Eric P.S. Baumer and M. Six Silberman. 2011. When the implication is not to design (technology). In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11, 2271–2274.

Digital Library

[6]

Jeremy Blackburn and Haewoon Kwak. 2014. STFU NOOB!: predicting crowdsourced decisions on toxic behavior in online games. In Proceedings of the 23rd international conference on World wide web - WWW ’14, 877–888.

Digital Library

[7]

Lindsay Blackwell, Jill Dimond, Sarita Schoenebeck, and Cliff Lampe. 2017. Classification and Its Consequences for Online Harassment: Design Insights from HeartMob. Proc. ACM Human-Computer Interact. 1, CSCW (2017), 24.

Digital Library

[8]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 2 (January 2006), 77–101.

[9]

Amy Bruckman, Pavel Curtis, Cliff Figallo, and Brenda Laurel. 1994. Approaches to managing deviant behavior in virtual communities. In Conference companion on Human factors in computing systems - CHI ’94, 183–184.

Digital Library

[10]

Kate Crawford and Tarleton Gillespie. 2016. What is a flag for? Social media reporting tools and the vocabulary of complaint. New Media Soc. 18, 3 (March 2016), 410–428.

[11]

Nicole Crenshaw, Jaclyn LaMorte, and Bonnie Nardi. 2017. "Something We Loved That Was Taken Away": Community and Neoliberalism in World of Warcraft. In Hawaii International Conference on System Sciences 2017 (HICSS-50). Retrieved May 30, 2020 from https://aisel.aisnet.org/hicss-50/dsm/games_and_gaming/5

[12]

Jessie Daniels. 2009. Cyber racism: White supremacy online and the new attack on civil rights. Rowman & Littlefield Publishers.

[13]

Jessie Daniels and Nick LaLone. 2012. Racism in video gaming: connecting extremist and mainstream expressions of white supremacy. Soc. exclusion, power, video game Play new Res. Digit. media Technol. (2012), 85–100.

[14]

Scott Donaldson. 2015. Mechanics and Metagame: Exploring Binary Expertise in League of Legends. Games Cult. (June 2015), 1–19.

[15]

Scott Donaldson. 2017. I Predict a Riot: Making and Breaking Rules and Norms in League of Legends. In DiGRA 2017. Retrieved June 1, 2020 from https://www.semanticscholar.org/paper/I-Predict-a-Riot%3A-Making-and-Breaking-Rules-and-in-Donaldson/99099cb226fb938ec35dfb124088151af9501afa

[16]

Bryan Dosono and Bryan Semaan. 2019. Moderation practices as emotional labor in sustaining online communities: The case of AAPI identity work on reddit. In Conference on Human Factors in Computing Systems - Proceedings.

Digital Library

[17]

Jenny Fan and Amy X. Zhang. 2020. Digital Juries: A Civics-Oriented Approach to Platform Governance. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–14.

Digital Library

[18]

Christopher Frauenberger, Amy S. Bruckman, Cosmin Munteanu, Melissa Densmore, and Jenny Waycott. 2017. Research Ethics in HCI: A Town Hall Meeting. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17, 1295–1299.

Digital Library

[19]

Guo Freeman and Donghee Yvette Wohn. 2017. eSports as An Emerging Research Context at CHI: Diverse Perspectives on Definitions. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17, 1601–1608.

Digital Library

[20]

Tarleton Gillespie. 2018. Custodians of the internet: platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.

[21]

Tarleton Gillespie. 2018. Facebook and YouTube just got more transparent. What do we see? niemanlab.

[22]

James Grimmelmann. 2015. The Virtues of Moderation. Yale J. Law Technol. 17, (2015). Retrieved September 2, 2019 from https://heinonline.org/HOL/Page?handle=hein.journals/yjolt17&id=42&div=3&collection=journals

[23]

Sidney V. Irwin and Anjum Naweed. 2018. BM'ing, Throwing, Bug Exploiting, and Other Forms of (Un)Sportsmanlike Behavior in CS:GO Esports. Games Cult. (October 2018), 155541201880495.

[24]

Shagun Jhaver, Iris Birman, Eric Gilbert, and Amy Bruckman. 2019. Human-Machine Collaboration for Content Regulation: The Case of Reddit Automoderator. ACM Trans. Comput. Interact. 26, 5 (July 2019), 1–35.

Digital Library

[25]

Shagun Jhaver, Amy Buckman, and Eric Gilbert. 2019. Does Transparency in Moderation Really Matter?: User Behavior After Content Removal Explanations on Reddit. Proc. ACM Human-Computer Interact. 3, CSCW (2019), 27.

Digital Library

[26]

Simon Joss and Arthur Brownlea. 1999. Considering the concept of procedural justice for public policy- and decision-making in science and technology. Sci. Public Policy 26, 5 (October 1999), 321–330.

[27]

Anja Kalch and Teresa K. Naab. 2017. Replying, disliking, flagging: How users engage with uncivil and impolite comments on news sites. Stud. Commun. | Media 6, 4 (January 2017), 395–419.

[28]

Sara Kiesler, Jane Siegel, and Timothy W. McGuire. 1984. Social psychological aspects of computer-mediated communication. Am. Psychol. 39 (1984), 1123–1134.

[29]

Yubo Kou. 2020. Toxic Behaviors in Team-Based Competitive Gaming: The Case of League of Legends. In Proceedings of the 2020 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY ’20, 81–92.

Digital Library

[30]

Yubo Kou and Xinning Gui. 2018. Entangled with Numbers: Quantified Self and Others in a Team-Based Online Game. Proc. ACM Human-Computer Interact. 2, CSCW (November 2018), 1–25.

Digital Library

[31]

Yubo Kou and Xinning Gui. 2020. Emotion Regulation in eSports Gaming: A Qualitative Study of League of Legends. Proc. ACM Human-Computer Interact. 4, CSCW2 (October 2020), 1–25.

Digital Library

[32]

Yubo Kou and Xinning Gui. 2020. Mediating Community-AI Interaction through Situated Explanation: The Case of AI-Led Moderation. Proc. ACM Human-Computer Interact. 4, CSCW2 (October 2020), 1–27.

Digital Library

[33]

Yubo Kou, Xinning Gui, and Yong Ming Kow. 2016. Ranking Practices and Distinction in League of Legends. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY ’16, 4–9.

Digital Library

[34]

Yubo Kou, Xinning Gui, Shaozeng Zhang, and Bonnie Nardi. 2017. Managing Disruptive Behavior through Non-Hierarchical Governance: Crowdsourcing in League of Legends and Weibo. Proc. ACM Human-Computer Interact. 1, CSCW (2017), 62.

Digital Library

[35]

Yubo Kou, Yao Li, Xinning Gui, and Eli Suzuki-Gill. 2018. Playing with Streakiness in Online Games: How Players Perceive and React to Winning and Losing Streaks in League of Legends. In CHI’2018, 1–14.

Digital Library

[36]

Yubo Kou and Bonnie Nardi. 2014. Governance in League of Legends: A Hybrid System. In Foundations of Digital Games.

[37]

Haewoon Kwak, Jeremy Blackburn, and Seungyeop Han. 2015. Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, 3739–3748.

Digital Library

[38]

Jean Lave and Etienne Wenger. 1991. Situated Learning: Legitimate Peripheral Participation (1st ed.). Cambridge University Press.

[39]

Lawrence Lessig. 2006. Code: And Other Laws of Cyberspace, Version 2.0. Basic Books.

Digital Library

[40]

Holin Lin and Chuen-Tsai Sun. 2005. The “White-Eyed” Player Culture: Grief Play and Construction of Deviance in MMORPGs. In DiGRA 2005.

[41]

Yang Liu, Qiaozhu Mei, David A Hanauer, Kai Zheng, and Joyce M Lee. 2016. Use of Social Media in the Diabetes Community: An Exploratory Analysis of Diabetes-Related Tweets. JMIR Diabetes 1, 2 (November 2016), e4.

[42]

Marçal Mora-Cantallops and Miguel-Ángel Sicilia. 2018. MOBA games: A literature review. Entertain. Comput. 26, (May 2018), 128–138.

[43]

Chip Morningstar and F. Randall Farmer. 1991. The Lessons of Lucasfilm's Habitat. In Cyberspace: First Steps, Michael Benedikt (ed.). MIT Press, Cambridge, MA, USA, 273–302. Retrieved from http://dl.acm.org/citation.cfm?id=114772.114792

Digital Library

[44]

Cosmin Munteanu, Amy Bruckman, Michael Muller, Christopher Frauenberger, Casey Fiesler, Robert E. Kraut, Katie Shilton, and Jenny Waycott. 2019. SIGCHI Research Ethics Town Hall. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA ’19, 1–6.

Digital Library

[45]

Teresa K Naab, Anja Kalch, and Tino GK Meitz. 2018. Flagging uncivil user comments: Effects of intervention information, type of victim, and response comments on bystander behavior. New Media Soc. 20, 2 (February 2018), 777–795.

[46]

Nikos Ntoumanis, Stuart J.H. Biddle, and Geoffrey Haddock. 1999. The mediating role of coping strategies on the relationship between achievement motivation and affect in sport. Anxiety, Stress Coping 12, 3 (1999), 299–327.

[47]

Patrick B. O'Sullivan and Andrew J. Flanagin. 2003. Reconceptualizing ‘flaming’ and other problematic messages. New Media Soc. 5, 1 (March 2003), 69–94.

[48]

Simon Parkin. 2015. A Video-Game Algorithm to Solve Online Abuse. MIT Technology Review`. Retrieved from https://www.technologyreview.com/s/541151/a-video-game-algorithm-to-solve-online-abuse/

[49]

Christopher A Paul. 2018. The Toxic Meritocracy of Video Games: Why Gaming Culture Is the Worst. University of Minnesota Press.

[50]

QSR International. 2019. NVivo 12. Retrieved from https://www.qsrinternational.com/nvivo/nvivo-products/nvivo-12-pro

[51]

Sarah T. Roberts. 2016. Commercial Content Moderation: Digital Laborers’ Dirty Work. Media Stud. Publ. (January 2016). Retrieved September 1, 2019 from https://ir.lib.uwo.ca/commpub/12

[52]

Kathleen Van Royen, Karolien Poels, and Heidi Vandebosch. 2016. Help, I Am Losing Control! Examining the Reporting of Sexual Harassment by Adolescents to Social Networking Sites. Cyberpsychology, Behav. Soc. Netw. 19, 1 (January 2016), 16–22.

[53]

Kenneth B. Shores, Yilin He, Kristina L. Swanenburg, Robert Kraut, and John Riedl. 2014. The identification of deviance and its impact on retention in a multiplayer game. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing - CSCW ’14, 1356–1365.

Digital Library

[54]

Spandana Singh. 2019. Everything in Moderation: An Analysis of How Internet Platforms Are Using Artificial Intelligence to Moderate User-Generated Content.

[55]

Lee Sproull and Sara Kiesler. 1986. Reducing Social Context Cues: Electronic Mail in Organizational Communication. Manag. Sci. 32, 11 (November 1986), 1492–1512.

[56]

Anselm L. Strauss. 1987. Qualitative analysis for social scientists. Cambridge University Press.

[57]

Mark C. Suchman. 1995. Managing Legitimacy: Strategic and Institutional Approaches. Acad. Manag. Rev. 20, 3 (July 1995), 571–610.

[58]

John Suler. 2004. The online disinhibition effect. Cyberpsychology Behav. impact Internet, Multimed. virtual Real. Behav. Soc. 7, 3 (June 2004), 321–326.

[59]

Pete Volk. 2016. League of Legends now boasts over 100 million monthly active players worldwide. riftherald.com. Retrieved from http://www.riftherald.com/2016/9/13/12865314/monthly-lol-players-2016-active-worldwide

[60]

Donghee Yvette Wohn. 2019. Volunteer moderators in twitch micro communities: How they get involved, the roles they play, and the emotional labor they experience. In Conference on Human Factors in Computing Systems - Proceedings.

Digital Library

[61]

YouTube. 2020. YouTube Trusted Flagger program. support.google.com. Retrieved from https://support.google.com/youtube/answer/7554338?hl=en

[62]

Bingjie Yu, Katta Spiel, and Leon Watts. Supporting Care as a Layer of Concern: Nurturing Attitudes in Online Community Moderation. Retrieved August 31, 2019 from https://www.metafilter.com

[63]

Amy X. Zhang, Grant Hugh, and Michael S. Bernstein. 2020. PolicyKit: Building Governance in Online Communities. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, 365–378.

Digital Library

Cited By

Marques TSchumann SMariconti E(2024)Positive behaviour interventions in online gaming: a systematic review of strategies applied in other environmentsCrime Science10.1186/s40163-024-00208-813:1Online publication date: 30-May-2024
https://doi.org/10.1186/s40163-024-00208-8
Meriläinen MRuotsalainen M(2024)Online Disinhibition, Normative Hostility, and Banal Toxicity: Young People’s Negative Online Gaming ConductSocial Media + Society10.1177/2056305124127466910:3Online publication date: 11-Sep-2024
https://doi.org/10.1177/20563051241274669
Lambert CChoi FChandrasekharan E(2024)"Positive reinforcement helps breed positive behavior": Moderator Perspectives on Encouraging Desirable BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/36869298:CSCW2(1-33)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686929
Show More Cited By

Index Terms

Flag and Flaggability in Automated Moderation: The Case of Reporting Toxic Behavior in an Online Game Community

Index terms have been assigned to the content through auto-classification.

Recommendations

Studying toxic behavior influence and player chat in an online video game
WI '17: Proceedings of the International Conference on Web Intelligence

Many online collaborative games, e-sports in particular, heavily rely on teamwork. However, players can act in an antisocial way during the match, creating dissent into the match. This kind of behavior is referred to as toxic. We aim to discover the ...
Toxic Behaviors in Team-Based Competitive Gaming: The Case of League of Legends
CHI PLAY '20: Proceedings of the Annual Symposium on Computer-Human Interaction in Play

Toxic behaviors in online gaming such as flaming and harassment have been gaining attention from the research community, yet little consensus has formed about what constitutes toxic behavior. Game developers usually maintain a classification system of ...
Prosocial behavior in an online game community: an ethnographic study
FDG '17: Proceedings of the 12th International Conference on the Foundations of Digital Games

We report an ethnographic study of prosocial behavior inconnection to League of Legends, one of the most popular games in the world. In this game community, the game developer, Riot Games, implemented a system that allowed players to volunteer their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
2,610
Total Downloads

Downloads (Last 12 months)865
Downloads (Last 6 weeks)136

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Marques TSchumann SMariconti E(2024)Positive behaviour interventions in online gaming: a systematic review of strategies applied in other environmentsCrime Science10.1186/s40163-024-00208-813:1Online publication date: 30-May-2024
https://doi.org/10.1186/s40163-024-00208-8
Meriläinen MRuotsalainen M(2024)Online Disinhibition, Normative Hostility, and Banal Toxicity: Young People’s Negative Online Gaming ConductSocial Media + Society10.1177/2056305124127466910:3Online publication date: 11-Sep-2024
https://doi.org/10.1177/20563051241274669
Lambert CChoi FChandrasekharan E(2024)"Positive reinforcement helps breed positive behavior": Moderator Perspectives on Encouraging Desirable BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/36869298:CSCW2(1-33)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686929
Sparrow LGalwey RJovic DHardwick TButt M(2024)Towards Ethical AI Moderation in Multiplayer GamesProceedings of the ACM on Human-Computer Interaction10.1145/36771098:CHI PLAY(1-30)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677109
Kordyaka BLaato SWeber SNiehaves B(2024)Are stress and engagement in toxicity associated with sleep quality? A study with League of Legends playersProceedings of the ACM on Human-Computer Interaction10.1145/36771018:CHI PLAY(1-17)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677101
Wijkstra M(2024)Fighting Toxicity Through Positive and Preventative InterventionCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678857(450-453)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3665463.3678857
Bongaards TAdriaanse MFrommel J(2024)Personalized Matchmaking Restrictions for Reduced Exposure to Toxicity: Preliminary Insights from an Interview StudyCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678803(31-36)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3665463.3678803
Huang ESarma AHwang SChandrasekharan EChancellor S(2024)Opportunities, tensions, and challenges in computational approaches to addressing online harassmentProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661623(1483-1498)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661623
Gunturi UKumar ADing XRho E(2024)Linguistically Differentiating Acts and Recalls of Racial Microaggressions on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/36373668:CSCW1(1-36)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637366
Frommel JMandryk R(2024)Toxicity in Online Games: The Prevalence and Efficacy of Coping StrategiesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642523(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642523
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents