abstract

Overview of the DravidianCodeMix 2021 Shared Task on Sentiment Detection in Tamil, Malayalam, and Kannada

Authors:

Ruba Priyadharshini,

Bharathi raja Chakravarthi,

Sajeetha Thavareesan,

Dhivya Chinnappa,

Durairaj Thenmozhi,

Rahul PonnusamyAuthors Info & Claims

FIRE '21: Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation

Pages 4 - 6

https://doi.org/10.1145/3503162.3503177

Published: 26 January 2022 Publication History

Abstract

We present the results of the Dravidian-CodeMix shared task1 held at FIRE 2021, a track on sentiment analysis for Dravidian Languages in Code-Mixed Text. We describe the task, its organization, and the submitted systems. This shared task is the continuation of last year’s Dravidian-CodeMix shared task2 held at FIRE 2020. This year’s tasks included code-mixing at the intra-token and inter-token levels. In addition to Tamil, Malayalam and Kannada were also introduced. We received 22 systems for Tamil-English, 15 systems for Malayalam-English, and 15 for Kannada-English. The top systems for Tamil-English, Malayalam-English and Kannada-English scored weighted average F1-score of 0.711, 0.804, and 0.630, respectively. In summary, the quality and quantity of the submission show that there is great interest in Dravidian languages in code-mixed setting and state of the art in this domain still needs improvement.

References

[1]

Bharathi B and Samyuktha G. U. 2021. Machine learning based approach for sentiment Analysis on Multilingual Code Mixing Text. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[2]

Yang Bai, Bangyuan Zhang, Yongjie Gu, Tongfeng Guan, and Qisong Shi. 2021. ZYBank-AI@Dravidian-CodeMix-FIRE2021: Automatic Detecting the Sentiment of Code-Mixed Text by Pre-training Model. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[3]

Fazlourrahman Balouchzahi, Hosahalli Lakshmaiah Shashirekha, and Grigori Sidorov. 2021. MUCIC@Dravidian-CodeMix-FIRE2021:CoSaD- Code-Mixed Sentiments Analysis for Dravidian Languages. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[4]

Bharathi Raja Chakravarthi. 2020. HopeEDI: A Multilingual Hope Speech Detection Dataset for Equality, Diversity, and Inclusion. In Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media. Association for Computational Linguistics, Barcelona, Spain (Online), 41–53. https://aclanthology.org/2020.peoples-1.5

[5]

Bharathi Raja Chakravarthi, Navya Jose, Shardul Suryawanshi, Elizabeth Sherly, and John Philip McCrae. 2020. A Sentiment Analysis Dataset for Code-Mixed Malayalam-English. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL). European Language Resources association, Marseille, France, 177–184. https://www.aclweb.org/anthology/2020.sltu-1.25

[6]

Bharathi Raja Chakravarthi, Vigneshwaran Muralidaran, Ruba Priyadharshini, and John Philip McCrae. 2020. Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL). European Language Resources association, Marseille, France, 202–210. https://www.aclweb.org/anthology/2020.sltu-1.28

[7]

Bharathi Raja Chakravarthi, Ruba Priyadharshini, Rahul Ponnusamy, Prasanna Kumar Kumaresan, Kayalvizhi Sampath, Durairaj Thenmozhi, S. Thangasamy, Rajendran Nallathambi, and John P. McCrae. 2021. Dataset for Identification of Homophobia and Transophobia in Multilingual YouTube Comments. ArXiv abs/2109.00227(2021).

[8]

Prasannakumaran D, Sideshwar J B, and Thenmozhi Durairaj. 2021. ECMAG - Ensemble of CNN and Multi-Head Attention with Bi-GRU for Sentiment Analysis in Code-Mixed Data. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[9]

Satyam Dutta, Himanshi Agrawal, and Pradeep Kumar Roy. 2021. DynamicDuo@Dravidian-CodeMix-FIRE2021: Sentiment Analysis on Multilingual Code Mixing Text using BERT. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[10]

Adeep Hande, Siddhanth U Hegde, Ruba Priyadharshini, Rahul Ponnusamy, Prasanna Kumar Kumaresan, Sajeetha Thavareesan, and Bharathi Raja Chakravarthi. 2021. Benchmarking Multi-Task Learning for Sentiment Analysis and Offensive Language Identification in Under-Resourced Dravidian Languages. ArXiv abs/2108.03867(2021).

[11]

Adeep Hande, Ruba Priyadharshini, and Bharathi Raja Chakravarthi. 2020. KanCMD: Kannada CodeMixed Dataset for Sentiment Analysis and Offensive Language Detection. In Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media. Association for Computational Linguistics, Barcelona, Spain (Online), 54–63. https://aclanthology.org/2020.peoples-1.6

[12]

Adeep Hande, R. Priyadharshini, Anbukkarasi Sampath, K. Thamburaj, Prabakaran Chandran, and Bharathi Raja Chakravarthi. 2021. Hope Speech detection in under-resourced Kannada language. ArXiv abs/2108.04616(2021).

[13]

Adeep Hande, Karthik Puranik, Konthala Yasaswini, Ruba Priyadharshini, Sajeetha Thavareesan, Anbukkarasi Sampath, Kogilavani Shanmugavadivel, Durairaj Thenmozhi, and Bharathi Raja Chakravarthi. 2021. Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling. ArXiv abs/2108.12177(2021).

[14]

Pawan Kalyan Jada, D Sashidhar Reddy, Konthala Yasaswini, Arunaggiri Pandian K, Prabakaran Chandran, Anbukkarasi Sampath, and Sathiyaraj Thangasamy. 2021. IIIT@Dravidian-CodeMix-FIRE2021: Transformer based Sentiment Analysis in Dravidian Languages. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[15]

A. Kalaivani and D. Thenmozhi. 2020. Multilingual Sentiment Analysis in Tamil, Malayalam, and Kannada code-mixed social media posts using MBERT. In FIRE (Working Notes).

[16]

Abhinav Kumar, Sunil Saumya, and Jyoti Prakash Singh. 2021. An ensemble-based model for sentiment analysis of Dravidian code-mixed social media posts. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[17]

Jyoti Kumari and Abhinav Kumar. 2021. A Deep Neural Network-based Model for the Sentiment Analysis of Dravidian Code-mixed Social Media Posts. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[18]

Anusha M D and Shasshirekha H L. 2021. MUM@Dravidian-CodeMix-FIRE2021:BiLSTM-Sentiments Analysis in Code MixedDravidian Languages. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[19]

Ankit Kumar Mishra, Sunil Saumya, and Abhinav Kumar. 2021. Sentiment Analysis of Dravidian-CodeMix Language. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[20]

Sripriya N and Divya S. 2021. SSN_IT_NLP@Dravidian-CodeMix-FIRE2021: Opinion And Attitude Investigation. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[21]

Varsha Pathak, Manish Joshi, Prasad Joshi, Monica Mundada, and Tanmay Joshi. 2020. KBCNMUJAL@HASOC-Dravidian-CodeMix-FIRE2020: Using Machine Learning for Detection of Hate Speech and Offensive Codemix Social Media text. In FIRE (Working Notes).

[22]

Pavan Kumar P.H.V, Premjith B, Sanjanasri Jp, and Soman Kp. 2021. Deep Learning Based Sentiment Analysis for Malayalam,Tamil and Kannada Languages. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[23]

Yandrapati Prakash Babu, Rajagopal Eswari, and K Nimmi. 2020. CIA_NITT@Dravidian-CodeMix-FIRE2020: Malayalam-English Code Mixed Sentiment Analysis Using Sentence BERT And Sentiment Features. In FIRE (Working Notes).

[24]

Karthik Puranik, Bharathi B, and B Senthil Kumar. 2021. Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[25]

Pradeep Kumar Roy and Abhinav Kumar. 2021. Sentiment Analysis on Tamil Code-Mixed Text using Bi-LSTM. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

[26]

Anita Saroj and Sukomal Pal. 2020. IRLab@IIT-BHU@Dravidian-CodeMix-FIRE2020: Sentiment Analysis on Multilingual Code Mixing Text Using BERT-BASE. In FIRE (Working Notes).

[27]

Sanjeepan Sivapiran, Charangan Vasantharajan, and Uthayasanker Thayasivam. 2021. RYZER@Dravidian-CodeMix-FIRE2021: Sentiment Analysis in Dravidian Code-Mixed YouTube Comments and Posts. In Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation (Online). CEUR.

Cited By

Prakash VVijay S(2025)An integrated framework for emotion and sentiment analysis in Tamil and Malayalam visual contentLanguage Resources and Evaluation10.1007/s10579-024-09804-1Online publication date: 5-Jan-2025
https://doi.org/10.1007/s10579-024-09804-1
C. VPandian S(2024) Fast Recurrent Neural Network with Bi-LSTM for Handwritten Tamil Text Segmentation in NLP ACM Transactions on Asian and Low-Resource Language Information Processing10.1145/364380823:5(1-20)Online publication date: 10-May-2024
https://dl.acm.org/doi/10.1145/3643808
Roy P(2024)Deep Ensemble Network for Sentiment Analysis in Bi-lingual Low-resource LanguagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/360022923:1(1-16)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3600229
Show More Cited By

Index Terms

Overview of the DravidianCodeMix 2021 Shared Task on Sentiment Detection in Tamil, Malayalam, and Kannada
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Language resources
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages and Conversational Hate Speech
FIRE '21: Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation

The HASOC track is dedicated to the evaluation of technology for finding Offensive Language and Hate Speech. HASOC is creating a multilingual data corpus mainly for English and under-resourced languages(Hindi and Marathi). This paper presents one HASOC ...
Findings of Shared Task on Offensive Language Identification in Tamil and Malayalam
FIRE '21: Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation

We present the results of HASOC-Dravidian-CodeMix shared task1 held at FIRE 2021, a track on offensive language identification for Dravidian languages in Code-Mixed Text in this paper. This paper will detail the task, its organisation, and the ...
Overview of the HASOC Track at FIRE 2020: Hate Speech and Offensive Language Identification in Tamil, Malayalam, Hindi, English and German
FIRE '20: Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation

This paper presents the HASOC track and its two parts. HASOC is dedicated to evaluate technology for finding Offensive Language and Hate Speech. HASOC is creating test collections for languages with few resources and English for comparison. The first ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

FIRE '21: Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation

December 2021

113 pages

ISBN:9781450395960

DOI:10.1145/3503162

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 January 2022

Check for updates

Author Tags

Qualifiers

Abstract
Research
Refereed limited

Conference

FIRE 2021

FIRE 2021: Forum for Information Retrieval Evaluation

December 13 - 17, 2021

Virtual Event, India

Acceptance Rates

Overall Acceptance Rate 19 of 64 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
136
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)14

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Prakash VVijay S(2025)An integrated framework for emotion and sentiment analysis in Tamil and Malayalam visual contentLanguage Resources and Evaluation10.1007/s10579-024-09804-1Online publication date: 5-Jan-2025
https://doi.org/10.1007/s10579-024-09804-1
C. VPandian S(2024) Fast Recurrent Neural Network with Bi-LSTM for Handwritten Tamil Text Segmentation in NLP ACM Transactions on Asian and Low-Resource Language Information Processing10.1145/364380823:5(1-20)Online publication date: 10-May-2024
https://dl.acm.org/doi/10.1145/3643808
Roy P(2024)Deep Ensemble Network for Sentiment Analysis in Bi-lingual Low-resource LanguagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/360022923:1(1-16)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3600229
Sathish VUmakanth KVarunsai MAlapati SLakshmi B(2024)Sentiment Analysis for Cross-Lingual Kannada–English Language PairProceedings of the Second International Conference on Computing, Communication, Security and Intelligent Systems10.1007/978-981-99-8398-8_11(165-173)Online publication date: 28-Mar-2024
https://doi.org/10.1007/978-981-99-8398-8_11
Elizabeth MKommineni AHazari R(2024)Sentiment Analysis for Code-Mixed Data Using Cellular Automata with Deep Learning ModelsCellular Automata10.1007/978-3-031-71552-5_14(163-176)Online publication date: 2-Sep-2024
https://doi.org/10.1007/978-3-031-71552-5_14
Saini JRoy S(2023)Preparation of Rich Lists of Research Gaps in the Specific Sentiment Analysis Tasks of Code-mixed Indian LanguagesSN Computer Science10.1007/s42979-023-02408-65:1Online publication date: 19-Dec-2023
https://dl.acm.org/doi/10.1007/s42979-023-02408-6
Shibly FSharma UNaleer H(2023)An Efficient Method for Detecting Hate Speech in Tamil Tweets Using an Ensemble ApproachInternational Conference on Innovative Computing and Communications10.1007/978-981-99-4071-4_2(19-26)Online publication date: 26-Oct-2023
https://doi.org/10.1007/978-981-99-4071-4_2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten