A Lexicon for Profane and Obscene Text Identification in Bengali

Proceedings of Recent Advances in Natural Language Processing, pages 1289–1296 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_145 1289 A Lexicon for Profane and Obscene Text Identification in Bengali Salim Sazzed Old Dominion University Norfolk, VA, USA ssazz001@odu.edu Abstract Bengali is a low-resource language that lacks tools and resources for profane and obscene textual content detection. Until now, no lexi- con exists for detecting obscenity in Bengali social media text. This study introduces a Bengali obscene lexicon consisting of over 200 Bengali terms that can be considered filthy, slang, profane or obscene. A semi- automatic methodology is presented for devel- oping the obscene lexicon that leverages an obscene corpus, word embedding, and part-of- speech (POS) taggers. The developed lexicon achieves coverage of around 0.85 for obscene and profane content detection in an evaluation dataset. The experimental results imply that the developed lexicon is effective at identify- ing obscenity in Bengali social media content. 1 Introduction The popularity of e-commerce and social media has surged the availability of user-generated content online. Therefore, text analysis tasks such as sen- timent classification (Feldman, 2013; Sazzed and Jayarathna, 2019; Yadollahi et al., 2017; Sazzed, 2021b; Sazzed and Jayarathna, 2021), hate speech detection (Poletto et al., 2021; Corazza et al., 2020), profane or abusive content identification (Caselli et al., 2020; Nobata et al., 2016) have received significant attention in recent years. Profanity in- dicates the usage of taboo or swearing words and is prevalent in social media data across languages (Wang et al., 2014). The presence of swearing, ob- scene or vulgar words could be linked with hate speech, sexism, and racism. Hence, identifying their presence is important to understanding and monitoring online content. Although the terms profanity, obscenity, swearing, and vulgarity have subtle differences in their meaning, they are closely connected with some overlapping definitions. Thus, in this paper, they have been used interchangeably to refer to filthy content. A lexicon consisting of a list of words with spe- cific annotations can play an important role in var- ious natural language processing tasks, such as sentiment analysis or inappropriate content iden- tification. A profane or obscene lexicon contains words that convey foul, filthy, and profane mean- ings (e.g., ass, bitch). An obscene lexicon is in- strumental for determining profanity, vulgarity, or obscenity in a text. The presence of swearing in En- glish social media has been investigated by various researchers (Wang et al., 2014; Pamungkas et al., 2020). Wang et al. (2014) found that the rate of swear word use in English Twitter is 1.15%, almost double compared to its use in daily conversation (0.5% - 0.7%) as observed in previous work (Jay, 1992). Wang et al. (2014) also reported that in a ran- dom sampling, they observed around 7.73% tweets containing swear words. Furthermore, vulgar word identification can help to improve sentiment classi- fication as shown by various researchers (Volkova et al., 2013; Yang and Eisenstein, 2017). In Bengali, although few works performed abu- sive content analysis, none of them focused on determining obscenity or generating resources …

Content truncated to 500 words