HateBR: A Large Expert Annotated Corpus of Brazilian Instagram Comments for Offensive Language and Hate Speech Detection

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7174–7183 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7174 HateBR: A Large Expert Annotated Corpus of Brazilian Instagram Comments for Offensive Language and Hate Speech Detection Francielle Vargas*†, Isabelle Carvalho*, Fabiana G´oes* Thiago A.S. Pardo*, Fabr´ıcio Benevenuto† *Institute of Mathematical and Computer Sciences, University of S˜ao Paulo, Brazil †Computer Science Department, Federal University of Minas Gerais, Brazil {francielleavargas,isabelle.carvalho,fabianagoes}@usp.br, taspardo@icmc.usp.br, fabricio@dcc.ufmg.br Abstract Due to the severity of the social media offensive and hateful comments in Brazil, and the lack of research in Portuguese, this paper provides the first large-scale expert annotated corpus of Brazilian Instagram comments for hate speech and offensive language detection. The HateBR corpus was collected from the comment section of Brazilian politicians’ accounts on Instagram and manually annotated by specialists, reaching a high inter-annotator agreement. The corpus consists of 7,000 documents annotated according to three different layers: a binary classification (offensive versus non-offensive comments), offensiveness-level classification (highly, moderately, and slightly offensive), and nine hate speech groups (xenophobia, racism, homophobia, sexism, religious intolerance, partyism, apology for the dictatorship, antisemitism, and fatphobia). We also implemented baseline experiments for offensive language and hate speech detection and compared them with a literature baseline. Results show that the baseline experiments on our corpus outperform the current state-of-the-art for the Portuguese language. Keywords: hate speech and offensive language detection, corpus annotation, natural language processing 1. Introduction Offensive language and hate speech detection has at- tracted interest from different institutions and has be- come an important research topic (Poletto et al., 2021; Pitenis et al., 2020; Zannettou et al., 2020; C¸ ¨oltekin, 2020; Guest et al., 2021). While this challenging en- deavour is undoubtedly a relevant research line, also has its implications for society concerning race, gender, religion, and origin. Furthermore, automated meth- ods for hateful and offensive comments detection may bolster web security in revealing individuals harboring malicious intentions towards specific groups (Gao et al., 2017). In Brazil, hate speech is prohibited, nevertheless the regulation is not effective due to the difficulty of iden- tifying, quantifying and classifying this kind of online content. The data on hate crimes in Brazil is very wor- risome: during the 2018 year’s election period, de- nunciations with xenophobia content had an increase of 2,369%; apology and public incitement to violence and crimes against life, 630%; neo-nazism, 548%; ho- mophobia, 350%; racism, 218%; and religious intoler- ance, 145% 1. The state-of-the-art has focused on different tasks, such as automatically detecting hate speech groups, for example, racism (Hasanuzzaman et al., 2017), anti- semitism (Zannettou et al., 2020), religious intolerance (Ghosh Chowdhury et al., 2019), misogyny and sexism (Guest et al., 2021; Jha and Mamidi, 2017), and cyber- bullying (Safi Samghabadi et al., 2020); filtering pages 1https://www.bbc.com/portuguese/ brasil-46146756 with hate and violence (Liu and Forss, 2015); offensive language detection (Zampieri et al., 2019; Steimel et al., 2019); and toxicity (Leite et al., 2020; Guimar˜aes et al., 2020). Schmidt and Wiegand (2017) present …

Content truncated to 500 words