research-article

Ex Machina: Personal Attacks Seen at Scale

Authors:

Ellery Wulczyn,

Nithum Thain, and

Lucas DixonAuthors Info & Claims

WWW '17: Proceedings of the 26th International Conference on World Wide Web

April 2017

Pages 1391 - 1399

https://doi.org/10.1145/3038912.3052591

Published: 03 April 2017 Publication History

Abstract

The damage personal attacks cause to online discourse motivates many platforms to try to curb the phenomenon. However, understanding the prevalence and impact of personal attacks in online platforms at scale remains surprisingly difficult. The contribution of this paper is to develop and illustrate a method that combines crowdsourcing and machine learning to analyze personal attacks at scale. We show an evaluation method for a classifier in terms of the aggregated number of crowd-workers it can approximate. We apply our methodology to English Wikipedia, generating a corpus of over 100k high quality human-labeled comments and 63M machine-labeled ones from a classifier that is as good as the aggregate of 3 crowd-workers, as measured by the area under the ROC curve and Spearman correlation. Using this corpus of machine-labeled scores, our methodology allows us to explore some of the open questions about the nature of online personal attacks. This reveals that the majority of personal attacks on Wikipedia are not the result of a few malicious users, nor primarily the consequence of allowing anonymous contributions from unregistered users.

References

[1]

J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. J. Mach. Learn. Res., 13:281--305, Feb. 2012.

[2]

M. Buhrmester, T. Kwang, and S. D. Gosling. Amazon's mechanical turk a new source of inexpensive, yet high-quality, data? Perspectives on psychological science, 6(1):3--5, 2011.

[3]

J. Cheng, C. Danescu-Niculescu-Mizil, and J. Leskovec. Antisocial behavior in online discussion communities. In ICWSM, 2015.

[4]

K. Dinakar, R. Reichart, and H. Lieberman. Modeling the detection of textual cyberbullying. The Social Mobile Web, 11:02, 2011.

[5]

M. Duggan. Online harassment. Pew Research Center, 2014.

[6]

Fundacion Barcelona Media (FBM). Caw 2.0 training datasets, 2009. http://caw2.barcelonamedia.org/.

[7]

I. Gagliardone, D. Gal, T. Alves, and G. Martinez. Countering online hate speech. UNESCO Publishing, 2015.

[8]

A. Halfaker. mwdiffs. https://github.com/mediawiki-utilities/python-mwdiffs.

[9]

A. F. Hayes and K. Krippendorff. Answering the call for a standard reliability measure for coding data. Communication methods and measures, 1(1):77--89, 2007.

[10]

Impermium. Detecting insults in social commentary dataset, 2012. https://www.kaggle.com/c/detecting-insults-in-social-commentary.

[11]

K. Krippendorff. Content analysis: An introduction to its methodology. Sage, 2004.

[12]

K. Krippendorff. Reliability in content analysis. Human communication research, 30(3):411--433, 2004.

[13]

I. Kwok and Y. Wang. Locate the hate: Detecting tweets against blacks. In AAAI, 2013.

Digital Library

[14]

M. J. Moore, T. Nakano, A. Enomoto, and T. Suda. Anonymity and roles associated with aggressive posts in an online forum. Computers in Human Behavior, 28(3):861--867, 2012.

Digital Library

[15]

C. Nobata, J. Tetreault, A. Thomas, Y. Mehdad, and Y. Chang. Abusive language detection in online user content. In WWW, 2016.

Digital Library

[16]

B. Pang and L. Lee. Opinion mining and sentiment analysis. Foundations and trends in information retrieval, 2(1--2):1--135, 2008.

Digital Library

[17]

S. Pieschl, C. Kuhlmann, and T. Porsch. Beware of publicity! perceived distress of negative cyber incidents and implications for defining cyberbullying. Journal of School Violence, 14(1):111--132, 2015.

[18]

B. Plank, D. Hovy, and A. Søgaard. Learning part-of-speech taggers with inter-annotator agreement loss. In EACL, pages 742--751, 2014.

[19]

H. M. Saleem, K. P. Dillon, S. Benesch, and D. Ruths. A web of hate: Tackling hateful speech in online social spaces. In TA-COS, 2016.

[20]

A. Schrock and D. Boyd. Problematic youth interaction online: Solicitation, harassment, and cyberbullying. Computer-Mediated Communication in Personal Relationships, pages 368--398, 2011.

[21]

S. O. Sood, E. F. Churchill, and J. Antin. Automatic identification of personal insults on social news sites. Journal of the American Society for Information Science and Technology, 63(2):270--285, 2012.

Digital Library

[22]

N. Spirin and J. Han. Survey on web spam detection: principles and algorithms. ACM SIGKDD Explorations Newsletter, 13(2):50--64, 2012.

Digital Library

[23]

Support and Safety Team. Harassment Survey. Wikimedia Foundation, 2015. https://upload.wikimedia.org/wikipedia/commons/5/52/Harassment_Survey_2015_-_Results_Report.pdf.

[24]

J. R. Tetreault, E. Filatova, and M. Chodorow. Rethinking grammatical error annotation and evaluation with the amazon mechanical turk. In NAACL-HLT, 2010.

Digital Library

[25]

R. S. Tokunaga. Following you home from school: A critical review and synthesis of research on cyberbullying victimization. Computers in human behavior, 26(3):277--287, 2010.

Digital Library

[26]

M. A. Walker, J. E. F. Tree, P. Anand, R. Abbott, and J. King. A corpus for research on deliberation and debate. In LREC, pages 812--817, 2012.

[27]

W. Warner and J. Hirschberg. Detecting hate speech on the world wide web. In LSM, 2012.

Digital Library

[28]

Z. Waseem and D. Hovy. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of NAACL-HLT, pages 88--93, 2016.

[29]

D. Wiener. Negligent publication of statements posted on electronic bulletin boards: Is there any liability left after zeran. Santa Clara L. Rev., 39:905, 1998.

[30]

Wikimedia. Harassment consultation 2015. https://meta.wikimedia.org/wiki/Harassment_consultation_2015.

[31]

Wikimedia. Machine-learning tool to reduce toxic talk page interactions. https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Bots_and_gadgets#Machine-learning_tool_to_reduce_toxic_talk_page_interactions.

[32]

Wikipedia. Help:Talk pages. https://www.mediawiki.org/wiki/Help:Talk_pages.

[33]

Wikipedia. Wikipedia:No personal attacks. https://en.wikipedia.org/wiki/Wikipedia:No_personal_attacks.

[34]

Wikipedia. Wikipedia:Revision_deletion. https://en.wikipedia.org/wiki/Wikipedia:Revision_deletion.

[35]

N. E. Willard. Cyberbullying and cyberthreats: Responding to the challenge of online social aggression, threats, and distress. Research Press, 2007.

Digital Library

[36]

E. Wulczyn, N. Thain, and L. Dixon. https://figshare.com/articles/Wikipedia_Detox_Data/4054689.

[37]

G. Xiang, B. Fan, L. Wang, J. Hong, and C. Rose. Detecting offensive tweets via topical feature discovery over a large scale twitter corpus. In CIKM, 2012.

Digital Library

[38]

J.-M. Xu, B. Burchfiel, X. Zhu, and A. Bellmore. An examination of regret in bullying tweets. In HLT-NAACL, pages 697--702, 2013.

[39]

M. L. Ybarra and K. J. Mitchell. Youth engaging in online harassment: Associations with caregiver--child relationships, internet use, and personal characteristics. Journal of adolescence, 27(3):319--336, 2004.

[40]

D. Yin, Z. Xue, L. Hong, B. D. Davison, A. Kontostathis, and L. Edwards. Detection of harassment on web 2.0. In WWW, 2009.

Cited By

Bulathwela SPérez-Ortiz MHolloway CCukurova MShawe-Taylor J(2024)Artificial Intelligence Alone Will Not Democratise Education: On Educational Inequality, Techno-Solutionism and Inclusive ToolsSustainability10.3390/su1602078116:2(781)Online publication date: 16-Jan-2024
https://doi.org/10.3390/su16020781
Bonechi S(2024)Development of an Automated Moderator for Deliberative EventsElectronics10.3390/electronics1303054413:3(544)Online publication date: 29-Jan-2024
https://doi.org/10.3390/electronics13030544
Yu ZSen IAssenmacher DSamory MFröhling LDahn CNozza DWagner C(2024)The Unseen Targets of Hate: A Systematic Review of Hateful Communication DatasetsSocial Science Computer Review10.1177/08944393241258771Online publication date: 13-Jun-2024
https://doi.org/10.1177/08944393241258771
Show More Cited By

Index Terms

Ex Machina: Personal Attacks Seen at Scale
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
2. Information systems
  1. Information systems applications
    1. Collaborative and social computing systems and tools
      1. Wikis

Recommendations

Students' Sense of Belonging in Online Classes: Does Age Matter?

This study explored graduate students' perceptions of their sense of belonging in online classes, their comfort in participating in online discussions, and their technology skills. Differences in these areas among three different age groups were ...
Read More
Investigating the Motivational Paths of Peer Production Newcomers
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Maintaining participation beyond the initial period of engagement is critical for peer production systems. Theory suggests that an increase in motivation is expected with contributors' movement from the community periphery to the core. Less is known, ...
Read More
Phantom Ex Machina: Digital Disruption's Role in Business Model Transformation
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '17: Proceedings of the 26th International Conference on World Wide Web

April 2017

1678 pages

ISBN:9781450349130

General Chairs:
Rick Barrett
W3Events
,
Rick Cummings
Murdoch University
,
Program Chairs:
Eugene Agichtein
Emory University
,
Evgeniy Gabrilovich
Google Research

Copyright © 2017 Copyright is held by the International World Wide Web Conference Committee (IW3C2).

Sponsors

IW3C2: International World Wide Web Conference Committee

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

International World Wide Web Conferences Steering Committee

Republic and Canton of Geneva, Switzerland

Publication History

Published: 03 April 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WWW '17

Sponsor:

IW3C2

WWW '17: 26th International World Wide Web Conference

April 3 - 7, 2017

Perth, Australia

Acceptance Rates

WWW '17 Paper Acceptance Rate 164 of 966 submissions, 17%;

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

265
Total Citations
View Citations
1,858
Total Downloads

Downloads (Last 12 months)236
Downloads (Last 6 weeks)21

Other Metrics

View Author Metrics

Citations

Cited By

Bulathwela SPérez-Ortiz MHolloway CCukurova MShawe-Taylor J(2024)Artificial Intelligence Alone Will Not Democratise Education: On Educational Inequality, Techno-Solutionism and Inclusive ToolsSustainability10.3390/su1602078116:2(781)Online publication date: 16-Jan-2024
https://doi.org/10.3390/su16020781
Bonechi S(2024)Development of an Automated Moderator for Deliberative EventsElectronics10.3390/electronics1303054413:3(544)Online publication date: 29-Jan-2024
https://doi.org/10.3390/electronics13030544
Yu ZSen IAssenmacher DSamory MFröhling LDahn CNozza DWagner C(2024)The Unseen Targets of Hate: A Systematic Review of Hateful Communication DatasetsSocial Science Computer Review10.1177/08944393241258771Online publication date: 13-Jun-2024
https://doi.org/10.1177/08944393241258771
Li LFan LAtreja SHemphill L(2024)“HOT” ChatGPT: The Promise of ChatGPT in Detecting and Discriminating Hateful, Offensive, and Toxic Comments on Social MediaACM Transactions on the Web10.1145/364382918:2(1-36)Online publication date: 2-Feb-2024
https://dl.acm.org/doi/10.1145/3643829
Gomez JMachado CPaes LCalmon F(2024)Algorithmic Arbitrariness in Content ModerationProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659036(2234-2253)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659036
Davani ADíaz MBaker DPrabhakaran V(2024)Disentangling Perceptions of Offensiveness: Cultural and Moral CorrelatesProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659021(2007-2021)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659021
Shawkat NSaquer JShatnawi HLo DGamess E(2024)Evaluation of Different Machine Learning and Deep Learning Techniques for Hate Speech DetectionProceedings of the 2024 ACM Southeast Conference10.1145/3603287.3651218(253-258)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3603287.3651218
Garg NSingh AManchanda M(2024)An Extensive Examination of Toxicity, Polarisation, and Biasing in Political Conversations on Social Media2024 IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT)10.1109/IC2PCT60090.2024.10486236(777-782)Online publication date: 9-Feb-2024
https://doi.org/10.1109/IC2PCT60090.2024.10486236
Teng TVarathan KCrestani F(2024)A comprehensive review of cyberbullying-related content classification in online social mediaExpert Systems with Applications10.1016/j.eswa.2023.122644244(122644)Online publication date: Jun-2024
https://doi.org/10.1016/j.eswa.2023.122644
Kebriaei EHomayouni AFaraji RRazavi AShakery AFaili HYaghoobzadeh Y(2024)Persian offensive language detectionMachine Language10.1007/s10994-023-06370-5113:7(4359-4379)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s10994-023-06370-5
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents