research-article

A Multilingual Evaluation for Online Hate Speech Detection

Authors:

Michele Corazza,

Stefano Menini,

Serena VillataAuthors Info & Claims

ACM Transactions on Internet Technology (TOIT), Volume 20, Issue 2

Article No.: 10, Pages 1 - 22

https://doi.org/10.1145/3377323

Published: 14 March 2020 Publication History

Abstract

The increasing popularity of social media platforms such as Twitter and Facebook has led to a rise in the presence of hate and aggressive speech on these platforms. Despite the number of approaches recently proposed in the Natural Language Processing research area for detecting these forms of abusive language, the issue of identifying hate speech at scale is still an unsolved problem. In this article, we propose a robust neural architecture that is shown to perform in a satisfactory way across different languages; namely, English, Italian, and German. We address an extensive analysis of the obtained experimental results over the three languages to gain a better understanding of the contribution of the different components employed in the system, both from the architecture point of view (i.e., Long Short Term Memory, Gated Recurrent Unit, and bidirectional Long Short Term Memory) and from the feature selection point of view (i.e., ngrams, social network–specific features, emotion lexica, emojis, word embeddings). To address such in-depth analysis, we use three freely available datasets for hate speech detection on social media in English, Italian, and German.

References

[1]

Sweta Agrawal and Amit Awekar. 2018. Deep learning for detecting cyberbullying across multiple social media platforms. In Proceedings of the 40th European Conference on IR Research—Advances in Information Retrieval (ECIR’18) (Lecture Notes in Computer Science), Gabriella Pasi, Benjamin Piwowarski, Leif Azzopardi, and Allan Hanbury (Eds.), Vol. 10772. Springer, 141--153.

[2]

Luis Enrique Argota Vega, Jorge Carlos Reyes-Magaña, Helena Gómez-Adorno, and Gemma Bel-Enguix. 2019. Detecting hate speech in Twitter using multiple features in a combinatorial framework. In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 447--452.

[3]

Pinar Arslan, Michele Corazza, Elena Cabrio, and Serena Villata. 2019. Overwhelmed by negative emotions? Maybe you are being cyber-bullied! In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (SAC’19).

[4]

Xiaoyu Bai, Flavio Merenda, Claudia Zaghi, Tommaso Caselli, and Malvina Nissim. 2018. Hate speech detection in Italian social media. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[5]

Xiaoyu Bai, Flavio Merenda, Claudia Zaghi, Tommaso Caselli, and Malvina Nissim. 2018. Detecting offensive speech in German social media. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[6]

Francesco Barbieri, Francesco Ronzano, and Horacio Saggion. 2016. What does this emoji mean? A vector space skip-gram model for Twitter emojis. In Proceedings of the Language Resources and Evaluation Conference (LREC’16).

[7]

Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. Multilingual detection of hate speech against immigrants and women in Twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics. 54--63.

[8]

Valerio Basile and Malvina Nissim. 2013. Sentiment analysis on Italian tweets. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 100--107.

[9]

Elisa Bassignana, Valerio Basile, and Viviana Patti. 2018. Hurtlex: A multilingual lexicon of words to hurt. In Proceedings of the 5th Italian Conference on Computational Linguistics (CLiC-it’18), Vol. 2253. CEUR-WS, 1--6.

[10]

Christos Baziotis, Nikos Pelekis, and Christos Doulkeridis. 2017. Deep LSTM with attention for message-level and topic-based sentiment analysis. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval’17). Association for Computational Linguistics, 747--754.

[11]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Trans. Assoc. Computat. Ling. 5 (2017), 135--146.

[12]

Cristina Bosco, Felice Dell’Orletta, Fabio Poletto, Manuela Sanguinetti, and Maurizio Tesconi. 2018. Overview of the EVALITA 2018 hate speech detection task. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[13]

Miguel Ángel Álvarez Carmona, Estefanía Guzmán-Falcón, Manuel Montes-y-Gómez, Hugo Jair Escalante, Luis Villaseñor Pineda, Verónica Reyes-Meza, and Antonio Rico Sulayes. 2018. Authorship and aggressiveness analysis in Mexican Spanish tweets. In Proceedings of the 3rd Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval’18) co-located with the 34th Conference of the Spanish Society for Natural Language Processing (SEPLN’18). 74--96.

[14]

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). Association for Computational Linguistics, 1724--1734.

[15]

François Chollet et al. 2015. Keras. Retrieved from https://github.com/fchollet/keras.

[16]

Mark Cieliebak, Jan Milan Deriu, Dominic Egger, and Fatih Uzdilli. 2017. A Twitter corpus and benchmark resources for German sentiment analysis. In Proceedings of the 5th International Workshop on Natural Language Processing for Social Media. Association for Computational Linguistics, 45--51.

[17]

Andrea Cimino, Lorenzo De Mattei, and Felice Dell’Orletta. 2018. Multi-task learning in deep neural networks at EVALITA 2018. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[18]

Michele Corazza, Stefano Menini, Pinar Arslan, Rachele Sprugnoli, Elena Cabrio, Sara Tonelli, and Serena Villata. 2018. Comparing different supervised approaches to hate speech detection. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[19]

Michele Corazza, Stefano Menini, Pinar Arslan, Rachele Sprugnoli, Elena Cabrio, Sara Tonelli, and Serena Villata. 2018. Identifying offensive tweets using recurrent neural networks. In Proceedings of the GermEval Workshop.

[20]

Michele Corazza, Stefano Menini, Elena Cabrio, Sara Tonelli, and Serena Villata. 2019. Cross-platform evaluation for Italian hate speech detection. In Proceedings of the 6th Italian Conference on Computational Linguistics. Retrieved from http://ceur-ws.org/Vol-2481/paper22.pdf.

[21]

Michele Corazza, Stefano Menini, Elena Cabrio, Sara Tonelli, and Serena Villata. 2019. InriaFBK drawing attention to offensive language at Germeval2019. In Proceedings of the 15th Conference on Natural Language Processing (KONVENS’19). Retrieved from https://corpora.linguistik.uni-erlangen.de/data/konvens/proceedings/papers/germeval/Germeval_Task_2_2019_paper_1.INRIA.pdf.

[22]

Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the 11th International Conference on Web and Social Media (ICWSM’17). 512--515.

[23]

Liangjie Hong, Brian D. Davison, April Kontostathis, Lynne Edwards, Dawei Yin, and Zhenzhen Xue. 2009. Detection of harassment on Web 2.0. In Proceedings of the Content Analysis in the Web Conference. 1--7.

[24]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19), Volume 1 (Long and Short Papers). 4171--4186. Retrieved from https://aclweb.org/anthology/papers/N/N19/N19-1423/.

[25]

Polina Stadnikova, Dietrich Klakow, Dominik Stammbach, and Azin Zahraei. 2018. Offensive language detection with neural networks for Germeval Task 2018. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[26]

Elisabetta Fersini, Paolo Rosso, and Maria Anzovino. 2018. Overview of the task on automatic misogyny identification at IberEval 2018. In Proceedings of the Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval@SEPLN’18) (CEUR Workshop Proceedings), Vol. 2150. CEUR-WS.org, 214--228.

[27]

Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Waseem, and Jacqueline Wernimont. 2018. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2’18). Association for Computational Linguistics.

[28]

Paula Fortuna, Ilaria Bonavita, and Sérgio Nunes. 2018. Merging datasets for hate speech classification in Italian. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[29]

Antigoni-Maria Founta, Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Athena Vakali, and Ilias Leontiadis. 2018. A unified deep learning architecture for abuse detection. CoRR abs/1802.00385 (2018). arxiv:1802.00385.

[30]

Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. Large scale crowdsourcing and characterization of Twitter abusive behavior. In Proceedings of the 12th International Conference on Web and Social Media (ICWSM’18). 491--500.

[31]

Björn Gambäck and Utpal Kumar Sikdar. 2017. Using convolutional neural networks to classify hate-speech. In Proceedings of the 1st Workshop on Abusive Language Online. Association for Computational Linguistics, 85--90. Retrieved from: http://aclweb.org/anthology/W17-3013.

[32]

Mario Graff, Sabino Miranda-Jiménez, Eric Sadit Tellez, Daniela Moctezuma, Vladimir Salgado, José Ortiz-Bejar, and Claudia N. Sánchez. 2018. Author profiling and aggressiveness analysis in Twitter using μTC and EvoMSA. In Proceedings of the 3rd Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval’18) co-located with the 34th Conference of the Spanish Society for Natural Language Processing (SEPLN’18). 128--133.

[33]

Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’18).

[34]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9, 8 (Nov. 1997), 1735--1780.

Digital Library

[35]

Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To Appear (2017).

[36]

Tianran Hu, Han Guo, Hao Sun, Thuy-vy Thi Nguyen, and Jiebo Luo. 2017. Spice up your chat: The intentions and sentiment effects of using emojis. In Proceedings of the 11th International Conference on Web and Social Media (ICWSM’17). 102--111.

[37]

Vijayasaradhi Indurthi, Bakhtiyar Syed, Manish Shrivastava, Nikhil Chakravartula, Manish Gupta, and Vasudeva Varma. 2019. Using sentence embeddings to identify hate speech against immigrants and women in Twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 70--74.

[38]

Rohan Kshirsagar, Tyrus Cukuvac, Kathy McKeown, and Susan McGregor. 2018. Predictive embeddings for hate speech detection on Twitter. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2’18). Association for Computational Linguistics, 26--32.

[39]

Gretel Liz De la Peña Sarracén, Reynaldo Gil Pons, Carlos Enrique Muñiz-Cuza, and Paolo Rosso. 2018. Hate speech detection using attention-based LSTM. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[40]

Younghun Lee, Seunghyun Yoon, and Kyomin Jung. 2018. Comparative studies of detecting abusive language on Twitter. CoRR abs/1808.10245 (2018). arxiv:1808.10245

[41]

Ping Liu, Wen Li, and Liang Zou. 2019. Transfer learning for offensive language detection using bidirectional transformers. In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 87--91.

[42]

Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’18).

[43]

Pushkar Mishra, Helen Yannakoudakis, and Ekaterina Shutova. 2018. Neural character-based composition models for abuse detection. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW’18). Association for Computational Linguistics, 1--10.

[44]

Saif M. Mohammad and Peter D. Turney. 2010. Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. Association for Computational Linguistics, 26--34.

Digital Library

[45]

Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word–emotion association lexicon. Computat. Intell. 29, 3 (2013), 436--465.

[46]

Roberto Navigli and Simone Paolo Ponzetto. 2012. BabelNet: The automatic construction, evaluation, and application of a wide-coverage multilingual semantic network. Artif. Intell. 193 (2012), 217--250.

Digital Library

[47]

Chikashi Nobata, Joel R. Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web (WWW’16). 145–153.

Digital Library

[48]

Joaquin Padilla Montani and Peter Schüller. 2018. German abusive tweet detection. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[49]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (2011), 2825--2830.

Digital Library

[50]

Juan Manuel Pérez and Franco M. Luque. 2019. Robust embeddings for tweet classification. In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 64--69.

[51]

Emanuele Pianta, Luisa Bentivogli, and Christian Girardi. 2002. MultiWordNet: Developing an aligned multilingual database. In Proceedings of the 1st International Conference on Global WordNet.

[52]

Marco Polignano and Pierpaolo Basile. 2018. HanSEL: Italian hate speech detection through ensemble learning and deep neural networks. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[53]

Valentino Santucci, Stefania Spina, Alfredo Milani, Giulio Biondi, and Gabriele Di Bari. 2018. Detecting hate speech for Italian language in social media. In Proceedings of the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA’18) co-located with the 5th Italian Conference on Computational Linguistics (CLiC-it’18).

[54]

Tatjana Scheffler, Erik Haegert, Santichai Pornavalaia, and Mino Lee Sasse. 2018. Feature explorations for hate speech classification. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[55]

Mike Schuster, Kuldip K. Paliwal, and A. General. 1997. Bidirectional recurrent neural networks. IEEE Trans. Sig. Proc. 45, 11 (1997), 2673--2681.

Digital Library

[56]

Abhishek Singh, Eduardo Blanco, and Wei Jin. 2019. Incorporating emoji descriptions improves tweet classification. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’19).

[57]

Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. 2017. Offline bilingual word vectors, orthogonal transformations, and the inverted softmax. In Proceedings of the 5th International Conference on Learning Representations (ICLR’17).

[58]

Samuel L. Smith, David H. P. Turban, Steven Hamblin, and Nils Y. Hammerla. 2017. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. CoRR abs/1702.03859 (2017). arxiv:1702.03859

[59]

Jacopo Staiano and Marco Guerini. 2014. Depeche mood: A lexicon for emotion analysis from crowd annotated news. In Proceedings of the 52nd Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 427--433.

[60]

Dirk von Grunigen, Ralf Grubenmann, Fernando Benites, Pius Von Daniken, and Mark Cieliebak. 2018. Classification of offensive content in tweets using convolutional neural networks and gated recurrent units. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[61]

Zeerak Waseem, Wendy Hui Kyong Chung, Dirk Hovy, and Joel Tetreault. 2017. In Proceedings of the 1st Workshop on Abusive Language Online. Association for Computational Linguistics.

[62]

Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In Proceedings of the Student Research Workshop in Conjunction with NAACL HLT 2018.

[63]

Gregor Wiedeman, Eugen Ruppert, Raghav Jindal, and Chris Biemann. 2018. Transfer learning from LDA to BiLSTM-CNN for offensive language detection in Twitter. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[64]

Michael Wiegand, Anastasija Amann, Tatiana Anikina, Aikaterini Azoidou, Anastasia Borisenkov, Kirstin Kolmorgen, Insa Kroger, and Christine Schafer. 2018. Examining different types of classifiers and features. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[65]

Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of abusive language: The problem of biased datasets. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 602--608.

[66]

Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the GermEval 2018 shared task on the identification of offensive language. In Proceedings of the GermEval 2018 and 14th Conference on Natural Language Processing (KONVENS’18).

[67]

Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex Machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web (WWW’17). 1391--1399.

Digital Library

[68]

Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Identifying and categorizing offensive language in social media (OffensEval). In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 75--86.

[69]

Robinson D. Zhang, Z. and J. Tepper. 2018. Detecting hate speech on Twitter using a convolution-GRU based deep neural network. In Proceedings of the Semantic Web Conference Proceedings (ESWC’18). Springer Verlag, 745--760.

Cited By

Bonechi S(2024)Development of an Automated Moderator for Deliberative EventsElectronics10.3390/electronics1303054413:3(544)Online publication date: 29-Jan-2024
https://doi.org/10.3390/electronics13030544
Mozafari MMnassri KFarahbakhsh RCrespi N(2024)Offensive language detection in low resource languages: A use case of Persian languagePLOS ONE10.1371/journal.pone.030416619:6(e0304166)Online publication date: 21-Jun-2024
https://doi.org/10.1371/journal.pone.0304166
Al Maruf AAbidin AHaque MJiyad ZGolder AAlubady RAung Z(2024)Hate speech detection in the Bengali language: a comprehensive surveyJournal of Big Data10.1186/s40537-024-00956-z11:1Online publication date: 23-Jul-2024
https://doi.org/10.1186/s40537-024-00956-z
Show More Cited By

Index Terms

A Multilingual Evaluation for Online Hate Speech Detection

Recommendations

Improving hate speech detection using Cross-Lingual Learning
Abstract
The growth of social media worldwide has brought social benefits and challenges. One problem we highlight is the proliferation of hate speech on social media. We propose a novel method for detecting hate speech in texts using Cross-Lingual ...
Highlights
- The development of a new methodology for hate speech detection.
- Portuguese hate speech detection using Cross-Lingual Learning.
- Up to 20% performance improvement over other models using the OffComBr-2 corpus.
HateCircle and Unsupervised Hate Speech Detection Incorporating Emotion and Contextual Semantics
The explosive growth of social media has fueled an extensive increase in online freedom of speech. The worldwide platform of human voice creates possibilities to assail other users without facing any consequences, and flout social etiquettes, resulting in ...
Multiword Expression Features for Automatic Hate Speech Detection
Natural Language Processing and Information Systems
Abstract
The task of automatically detecting hate speech in social media is gaining more and more attention. Given the enormous volume of content posted daily, human monitoring of hate speech is unfeasible. In this work, we propose new word-level features ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology

ACM Transactions on Internet Technology Volume 20, Issue 2

Special Section on Emotions in Conflictual Social Interactions and Regular Papers

May 2020

256 pages

ISSN:1533-5399

EISSN:1557-6051

DOI:10.1145/3386441

Editor:
Ling Liu
Georgia Institute of Technology, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2020

Accepted: 01 December 2019

Revised: 01 December 2019

Received: 01 March 2019

Published in TOIT Volume 20, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

CREEP project
HATEMETER project
EIT Digital in 2018 and 2019
EU Rights, Equality and Citizenship Programme 2014–2020

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

102
Total Citations
View Citations
2,670
Total Downloads

Downloads (Last 12 months)266
Downloads (Last 6 weeks)26

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bonechi S(2024)Development of an Automated Moderator for Deliberative EventsElectronics10.3390/electronics1303054413:3(544)Online publication date: 29-Jan-2024
https://doi.org/10.3390/electronics13030544
Mozafari MMnassri KFarahbakhsh RCrespi N(2024)Offensive language detection in low resource languages: A use case of Persian languagePLOS ONE10.1371/journal.pone.030416619:6(e0304166)Online publication date: 21-Jun-2024
https://doi.org/10.1371/journal.pone.0304166
Al Maruf AAbidin AHaque MJiyad ZGolder AAlubady RAung Z(2024)Hate speech detection in the Bengali language: a comprehensive surveyJournal of Big Data10.1186/s40537-024-00956-z11:1Online publication date: 23-Jul-2024
https://doi.org/10.1186/s40537-024-00956-z
Kumar S(2024)Negative Stances Detection from Multilingual Data Streams in Low-Resource Languages on Social Media Using BERT and CNN-Based Transfer Learning ModelACM Transactions on Asian and Low-Resource Language Information Processing10.1145/362582123:1(1-18)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3625821
De Oliveira ABaptista CFirmino ADe Paiva AHong JPark J(2024)A Large Language Model Approach to Detect Hate Speech in Political Discourse Using Multiple Language CorporaProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635964(1461-1468)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3635964
Shahrezaie RHajibabaee PAhmadi M(2024)An Open Source Framework for Benchmarking Cyberbullying Detection on Social Media2024 IEEE World AI IoT Congress (AIIoT)10.1109/AIIoT61789.2024.10578983(471-478)Online publication date: 29-May-2024
https://doi.org/10.1109/AIIoT61789.2024.10578983
Altinel ABaydogmus GSahin SGurbuz M(2024)So-haTRed: A Novel Hybrid System for Turkish Hate Speech Detection in Social Media With Ensemble Deep Learning Improved by BERT and Clustered-Graph NetworksIEEE Access10.1109/ACCESS.2024.341535012(86252-86270)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3415350
P.Y.K.L SDas APulabaigari V(2024)Racists spreader is narcissistic; sexists is Machiavellian Influence of Psycho-Sociological Facets in hate-speech diffusion predictionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123211247:COnline publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123211
Mahajan EMahajan HKumar S(2024)EnsMulHateCyb: Multilingual hate speech and cyberbully detection in online social mediaExpert Systems with Applications10.1016/j.eswa.2023.121228236(121228)Online publication date: Feb-2024
https://doi.org/10.1016/j.eswa.2023.121228
Firmino Ade Souza Baptista Cde Paiva A(2024)Improving hate speech detection using Cross-Lingual LearningExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121115235:COnline publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.121115
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents