Inducing a Lexicon of Abusive Words – a Feature-Based Approach

Proceedings of NAACL-HLT 2018, pages 1046–1056 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Inducing a Lexicon of Abusive Words – A Feature-Based Approach Michael Wiegand∗, Josef Ruppenhofer†, Anna Schmidt∗, Clayton Greenberg∗ ∗Spoken Language Systems, Saarland University, D-66123 Saarbr¨ucken, Germany †Institute for German Language, D-68161 Mannheim, Germany michael.wiegand@lsv.uni-saarland.de ruppenhofer@ids-mannheim.de anna.schmidt@lsv.uni-saarland.de clayton.greenberg@lsv.uni-saarland.de Abstract We address the detection of abusive words. The task is to identify such words among a set of negative polar expressions. We propose novel features employing information from both corpora and lexical resources. These fea- tures are calibrated on a small manually an- notated base lexicon which we use to produce a large lexicon. We show that the word-level information we learn cannot be equally de- rived from a large dataset of annotated mi- croposts. We demonstrate the effectiveness of our (domain-independent) lexicon in the cross- domain detection of abusive microposts. 1 Introduction Abusive or offensive language is commonly de- fined as hurtful, derogatory or obscene utterances made by one person to another person.1 Examples are (1)-(3). In the literature, closely related terms include hate speech (Waseem and Hovy, 2016) or cyber bullying (Zhong et al., 2016). While there may be nuanced differences in meaning2, they are all compatible with the general definition above for abusive language.3 (1) stop editing this, you dumbass. (2) Just want to slap the stupid out of these bimbos!!! (3) Go lick a pig you arab muslim piece of scum. Due to the rise of user-generated web content, in particular on social media networks, the amount of abusive language is also steadily growing. NLP methods are required to focus human review ef- forts towards the most relevant microposts. In this paper, we address the task of detecting abusive words (e.g. dumbass, bimbo, scum). Our 1http://thelawdictionary.org/ 2For example, several research efforts just focus on utter- ances addressed towards minorities. 3The examples in this work are included to illustrate the severity of abusive language. They are taken from actual web data and in no way reflect the opinion of the authors. main assumption is that abusive words form a sub- set of negative polar expressions. The classifi- cation task is to filter the abusive words from a given set of negative polar expressions. We proceed as follows. On a base lexicon that is a small subset of negative polar expressions where the abusive words among them have been marked via crowdsourcing (§3), we calibrate a super- vised classifier by examining various novel fea- tures (§4). A classifier trained on that base lex- icon, which contains 551 abusive words, is then applied to a very large list of unlabeled negative polar expressions (from Wiktionary) to extract an expanded lexicon of 2989 abusive words (§5). We extrinsically evaluate our new lexicon in the novel task of cross-domain classification of abu- sive documents (§6) where we use it as a high- level feature. In this work, we consider microp- osts as documents. While for in-domain classi- fication, supervised classifiers trained on generic features, …

Content truncated to 500 words