Improving Hate Speech Detection with Deep Learning Ensembles

Improving Hate Speech Detection with Deep Learning Ensembles Steven Zimmerman, Chris Fox, Udo Kruschwitz University of Essex Wivenhoe Park, Colchester CO4 3SQ, United Kingdom {szimme, foxcj, udo}@essex.ac.uk Abstract Hate speech has become a major issue that is currently a hot topic in the domain of social media. Simultaneously, current proposed methods to address the issue raise concerns about censorship. Broadly speaking, our research focus is the area human rights, including the development of new methods to identify and better address discrimination while protecting freedom of expression. As neural network approaches are becoming state of the art for text classification problems, an ensemble method is adapted for usage with neural networks and is presented to better classify hate speech. Our method utilizes a publicly available embedding model, which is tested against a hate speech corpus from Twitter. To confirm robustness of our results, we additionally test against a popular sentiment dataset. Given our goal, we are pleased that our method has a nearly 5 point improvement in F-measure when compared to original work on a publicly available hate speech evaluation dataset. We also note difficulties encountered with reproducibility of deep learning methods and comparison of findings from other work. Based on our experience, more details are needed in published work reliant on deep learning methods, with additional evaluation information a consideration too. This information is provided to foster discussion within the research community for future work. Keywords: Hate Speech, Reproducibility, Text Classification 1. Introduction and Motivation Our research is focused on the development of better meth- ods for protection of freedom of expression in the web do- main and social media while simultaneously reducing ille- gal discrimination. Motivation is provided by the funda- mental human rights (as outlined in articles 19 and 20 of (The United Nations, 1948) and (The United Nations Gen- eral Assembly, 1966)) which simultaneously provide rights to freedom of expression and prevent censorship and illegal discrimination. Automated take down approaches poten- tially infringe upon rights to freedom of expression, such as when a text classifier incorrectly flags a page or post as something to be taken down. Hate speech classifiers are based on annotation methods that are very difficult to de- fine, with questionable reliability (Ross et al., 2017). Even a manual take down approach, such as that used by Face- book, is a challenging task1. Censorship is a potential risk when addressing these issues with automated text classification methods, thus all options should be considered (Benesch, 2017). Actions to filter and block content (e.g. recently implemented laws in Germany and by platforms such as Twitter and Facebook) deemed to be hateful and / or threatening to the online community and society as whole have been taken, which is having negative consequences2. The goal of our work is to discover simple but effective methods to improve upon existing research in the area of hate speech classification. These methods will be useful in our broader research which tests mechanisms that provide users with feedback about their consumption of …

Content truncated to 500 words