Hate speech detection: a comparison of mono and multilingual transformer model with cross-language evaluation

Hate speech detection: a comparison of mono and multilingual transformer model with cross-language evaluation Koyel Ghosh Central Institute of Technology, Kokrajhar, Assam, India ghosh.koyel8@gmail.com Apurbalal Senapati Central Institute of Technology, Kokrajhar, Assam, India a.senapati@cit.ac.in Abstract Warning: This paper contains examples of the language that some people may find offensive. Transformer-based Language models have achieved state-of-the-art performance on a wide range of Natural Language Process- ing (NLP) tasks. This work will examine the effectiveness of transformer language models like BERT, RoBERTa, ALBERT, and DistilBERT on existing Indian hate speech datasets such as HASOC-Hindi (2019), HASOC-Marathi (2021) and Ben- gali Hate Speech (BenHateSpeech) over binary classification. Most deep learning methods fail to recognize a hate sentence if hate words are wrapped into sophis- ticated words where transformers under- stand the context of a hate word present in a sentence. Here, Transformer-based multilingual models such as MuRILBERT, XLM-RoBERTa, etc. are compared with monolingual models like NeuralSpace- BERTHi (Hindi), MahaBERT (Marathi), BanglaBERT (Bengali), etc. It is no- ticed that the monolingual MahaBERT model performs the best on HASOC- Marathi, whereas the multilingual MuRIL- BERT performs the best on HASOC-Hindi and BenHateSpeech. Several other cross- language evaluations over Marathi and Hindi monolingual models and mixed ob- servations are presented. 1 Introduction According to the Cambridge Dictionary, hate speech is defined as - “Hate speech is a public speech that expresses hate or encourages vio- lence towards a person or group based on race, religion, sex, or sexual orientation”1. Statis- tics reveal that half of the world’s population, including print media, is now engaged in so- cial media platforms2 and 12½ trillion hours spent online by the users3. This trend shall continue till obvious infinity. Sometimes ag- gressive posts, misleading news, and harassing comments can lead people to social violence, even riots (Laub, 2019). Worldwide, Govern- ments are introducing laws against hate speech. So, digital media like Twitter, Facebook, etc., are also becoming more concerned about it and endeavouring to filter hate, sexual abuse, harm- ful acts, harassment, bullying, child abuse, etc. Researchers explore this field, but most of the experiment is based on European language datasets4. Limited work is done on the In- dian languages except for publishing datasets or improving accuracy. India has 22 official languages and about 1,000 living languages from various language groups (Kalra and Dutt, 2019). People in India use their native lan- 1https://dictionary.cambridge.org/dictionary/english/hate-speech 2https://datareportal.com/reports/digital-2021-global-overview-report 3https://datareportal.com/reports/digital-2022-global-overview-report 4https://hatespeechdata.com/ guages on social media platforms, and some- times users don’t follow the proper structure or grammar, making it more complicated to detect hate speech in the computational aspect. This situation motivated us to work on hate speech on Twitter and other social media texts. It is challenging for automatic approaches to detect hate speech in text. Researchers use state-of-the-art transformer models more in language-related researchs like NLP, Information Retrieval (IR), etc., to enrich performance. Many works have already been done in NLP like text classification (Sun et al., 2019), question-answering (McCarley et al., 2019), token classification (Ulˇcar and Robnik- ˇSikonja, 2020), and Named Entity …

Content truncated to 500 words