research-article

Individual vs. Group Violent Threats Classification in Online Discussions

Authors:

Grigori Sidorov,

Alexander GelbukhAuthors Info & Claims

WWW '20: Companion Proceedings of the Web Conference 2020

Pages 629 - 633

https://doi.org/10.1145/3366424.3385778

Published: 20 April 2020 Publication History

Abstract

Violent threat is a serious crime affecting the targeted individuals or groups. It is essential for media providers to block the users that post such threats. In this paper, we focused on detection of violent threat language in YouTube comments. We categorized the threatening comments into those targeting an individual or a group. We started from an existing dataset with violent threat language identified, but without any categorization into comments targeting individuals or groups. We adopted a binary classification approach for the prediction of individual- vs. group-targeting threats. We compared two text representations: bag of words (BOW) and pre-trained word embedding such as GloVe and fastText. We used deep-learning classifiers such as 1D-CNN, LSTM, and bidirectional LSTM (BiLSTM). GloVe embedding showed the worst results, fastText performed much better, and BiLSTM on BOW with term frequency-inverse document frequency (TF-IDF) weighting scheme gave the best results, achieving 0.94% ROC-AUC and Macro-F1 score of 0.85%.

References

[1]

Swati Agarwal and Ashish Sureka. 2015. Using KNN and SVM based one-class classifier for detecting online radicalization on Twitter. In International Conference on Distributed Computing and Internet Technology. Springer, 431–442.

Digital Library

[2]

Ayyaz Yaqoob Abid Rafiq Bashir Farhan, Ashraf Noman and Raza Ul Mustafa. 2019. Human aggressiveness and reactions towards uncertain decisions. International Journal of Advanced and Applied Sciences 6, 7 (2019), 112–116.

[3]

Karthik Dinakar, Roi Reichart, and Henry Lieberman. 2011. Modeling the detection of textual cyberbullying. In fifth international AAAI conference on weblogs and social media.

[4]

Iginio Gagliardone, Danit Gal, Thiago Alves, and Gabriela Martinez. 2015. Countering online hate speech. Unesco Publishing.

[5]

Hugo L Hammer, Michael A Riegler, Lilja Øvrelid, and Erik Velldal. 2019. THREAT: A Large Annotated Corpus for Detection of Violent Threats. In 2019 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 1–5.

[6]

Sepp Hochreiter. 1998. The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6, 02 (1998), 107–116.

Digital Library

[7]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.

Digital Library

[8]

Linda Camp Keith. 1999. The United Nations International Covenant on Civil and Political Rights: Does it make a difference in human rights behavior?Journal of Peace Research 36, 1 (1999), 95–118.

[9]

Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1746–1751. https://doi.org/10.3115/v1/D14-1181

[10]

Varada Kolhatkar and Maite Taboada. 2017. Constructive language in news comments. In Proceedings of the First Workshop on Abusive Language Online. 11–17.

[11]

Xuan-Hien Le, Hung Viet Ho, Giha Lee, and Sungho Jung. 2019. Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting. Water 11, 7 (2019), 1387.

[12]

Courtney Napoles, Joel Tetreault, Aasish Pappu, Enrica Rosato, and Brian Provenzale. 2017. Finding good conversations online: The yahoo news annotated comments corpus. In Proceedings of the 11th Linguistic Annotation Workshop. 13–23.

[13]

Nelleke Oostdijk and Hans van Halteren. 2013a. N-gram-based recognition of threatening tweets. In International Conference on Intelligent Text Processing and Computational Linguistics. Springer, 183–196.

Digital Library

[14]

Nelleke Oostdijk and Hans van Halteren. 2013b. Shallow parsing for recognizing threats in Dutch tweets. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining. 1034–1041.

Digital Library

[15]

Juan Ramos. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, Vol. 242. Piscataway, NJ, 133–142.

[16]

William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the second workshop on language in social media. Association for Computational Linguistics, 19–26.

Digital Library

[17]

Aksel Wester, Lilja Øvrelid, Erik Velldal, and Hugo Lewi Hammer. 2016. Threat detection in online discussions. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 66–71.

[18]

Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the GermEval 2018 shared task on the identification of offensive language. In Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018). Vienna, Austria, 1–10.

[19]

Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web. 1391–1399.

Digital Library

Cited By

Bifari EBasbrain AMirza RBafail AAlbaradei SAlhalabi W(2024)Text mining and machine learning for crime classification: using unstructured narrative court documents in police academicCogent Engineering10.1080/23311916.2024.235985011:1Online publication date: 3-Jun-2024
https://doi.org/10.1080/23311916.2024.2359850
Nazarova AMalik MIgnatov DHussain I(2024)Deepthreatexplainer: a united explainable predictor for threat comments identification on TwitterSocial Network Analysis and Mining10.1007/s13278-024-01389-514:1Online publication date: 3-Dec-2024
https://doi.org/10.1007/s13278-024-01389-5
Malik M(2024)Threatening Expression and Target Identification in Under-Resource Languages Using NLP TechniquesAnalysis of Images, Social Networks and Texts10.1007/978-3-031-54534-4_1(3-17)Online publication date: 12-Mar-2024
https://doi.org/10.1007/978-3-031-54534-4_1
Show More Cited By

Index Terms

Individual vs. Group Violent Threats Classification in Online Discussions

Index terms have been assigned to the content through auto-classification.

Recommendations

“We found no violation!”: Twitter's Violent Threats Policy and Toxicity in Online Discourse
C&T '21: Proceedings of the 10th International Conference on Communities & Technologies - Wicked Problems in the Age of Tech

Threat moderation on social media has been subject to much public debate and criticism, especially for its broadly permissive approach. In this paper, we focus on Twitter's Violent Threats policy, highlighting its shortcomings by comparing it to ...
Detecting Threats of Violence in Online Discussions Using Bigrams of Important Words
JISIC '14: Proceedings of the 2014 IEEE Joint Intelligence and Security Informatics Conference

Making violent threats towards minorities like immigrants or homosexuals is increasingly common on the Internet. We present a method to automatically detect threats of violence using machine learning. A material of 24,840 sentences from YouTube was ...
Sentimental Short Sentences Classification by Using CNN Deep Learning Model with Fine Tuned Word2Vec
Abstract
Continues growth of social networking web users, people daily shared their ideas and opinions in the form of texts, images, videos, and speech. Text categorization is still a crucial issue because these huge texts received from the heterogeneous ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Companion Proceedings of the Web Conference 2020

April 2020

854 pages

ISBN:9781450370240

DOI:10.1145/3366424

Editors:
Amal El Fallah Seghrouchni
Sorbonne University, France
,
Gita Sukthankar
University of Central Florida, United States
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
228
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)3

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bifari EBasbrain AMirza RBafail AAlbaradei SAlhalabi W(2024)Text mining and machine learning for crime classification: using unstructured narrative court documents in police academicCogent Engineering10.1080/23311916.2024.235985011:1Online publication date: 3-Jun-2024
https://doi.org/10.1080/23311916.2024.2359850
Nazarova AMalik MIgnatov DHussain I(2024)Deepthreatexplainer: a united explainable predictor for threat comments identification on TwitterSocial Network Analysis and Mining10.1007/s13278-024-01389-514:1Online publication date: 3-Dec-2024
https://doi.org/10.1007/s13278-024-01389-5
Malik M(2024)Threatening Expression and Target Identification in Under-Resource Languages Using NLP TechniquesAnalysis of Images, Social Networks and Texts10.1007/978-3-031-54534-4_1(3-17)Online publication date: 12-Mar-2024
https://doi.org/10.1007/978-3-031-54534-4_1
Shrestha AKaati LAkrami NLinden KMoshfegh ARokne JWang D(2023)Harmful Communication: Detection of Toxic Language and Threats on SwedishProceedings of the International Conference on Advances in Social Networks Analysis and Mining10.1145/3625007.3627597(624-630)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625007.3627597
Qian WYu SNie ZLu XLiu HHuang B(2023)Improved Hierarchical Attention Networks for Cyberbullying Detection via Social Media Data2023 IEEE International Conference on Networking, Sensing and Control (ICNSC)10.1109/ICNSC58704.2023.10319023(1-6)Online publication date: 25-Oct-2023
https://doi.org/10.1109/ICNSC58704.2023.10319023
Rehan MMalik MJamjoom M(2023)Fine-Tuning Transformer Models Using Transfer Learning for Multilingual Threatening Text IdentificationIEEE Access10.1109/ACCESS.2023.332006211(106503-106515)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3320062
Prause NLey D(2023)Violence on Reddit Support Forums Unique to r/NoFapDeviant Behavior10.1080/01639625.2023.228079545:4(602-618)Online publication date: 15-Nov-2023
https://doi.org/10.1080/01639625.2023.2280795
Amjad AKhan LAshraf NMahmood MChang H(2022)Recognizing Semi-Natural and Spontaneous Speech Emotions Using Deep Neural NetworksIEEE Access10.1109/ACCESS.2022.316371210(37149-37163)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3163712
Khan LAmjad AAshraf NChang H(2022)Multi-class sentiment analysis of urdu text using multilingual BERTScientific Reports10.1038/s41598-022-09381-912:1Online publication date: 31-Mar-2022
https://doi.org/10.1038/s41598-022-09381-9
Adebanji OGelbukh ICalvo HOjo O(2022)Sequential Models for Sentiment Analysis: A Comparative StudyAdvances in Computational Intelligence10.1007/978-3-031-19496-2_17(227-235)Online publication date: 23-Oct-2022
https://doi.org/10.1007/978-3-031-19496-2_17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents