Deeper Attention to Abusive User Content Moderation

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1125–1135 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Deeper Attention to Abusive User Content Moderation John Pavlopoulos Straintek, Athens, Greece ip@straintek.com Prodromos Malakasiotis Straintek, Athens, Greece mm@straintek.com Ion Androutsopoulos Athens University of Economics and Business, Greece ion@aueb.gr Abstract Experimenting with a new dataset of 1.6M user comments from a news portal and an existing dataset of 115K Wikipedia talk page comments, we show that an RNN op- erating on word embeddings outpeforms the previous state of the art in moderation, which used logistic regression or an MLP classifier with character or word n-grams. We also compare against a CNN operat- ing on word embeddings, and a word-list baseline. A novel, deep, classification- specific attention mechanism improves the performance of the RNN further, and can also highlight suspicious words for free, without including highlighted words in the training data. We consider both fully auto- matic and semi-automatic moderation. 1 Introduction User comments play a central role in social me- dia and online discussion fora. News portals and blogs often also allow their readers to com- ment to get feedback, engage their readers, and build customer loyalty.1 User comments, how- ever, and more generally user content can also be abusive (e.g., bullying, profanity, hate speech) (Cheng et al., 2015). Social media are under pres- sure to combat abusive content, but so far rely mostly on user reports and tools that detect fre- quent words and phrases of reported posts.2 Wul- czyn et al. (2017) estimated that only 17.9% of personal attacks in Wikipedia discussions were followed by moderator actions. News portals also 1 See, for example, http://niemanreports.org/ articles/the-future-of-comments/. 2 Consult, for example, https://www.facebook. com/help/131671940241729 and https://www. theguardian.com/technology/2017/feb/07/ twitter-abuse-harassment-crackdown. suffer from abusive user comments, which dam- age their reputations and make them liable to fines, e.g., when hosting comments encouraging illegal actions. They often employ moderators, who are frequently overwhelmed, however, by the volume and abusiveness of comments.3 Readers are dis- appointed when non-abusive comments do not ap- pear quickly online because of moderation delays. Smaller news portals may be unable to employ moderators, and some are forced to shut down their comments sections entirely. We examine how deep learning (Goodfellow et al., 2016; Goldberg, 2016, 2017) can be em- ployed to moderate user comments. We experi- ment with a new dataset of approx. 1.6M manually moderated (accepted or rejected) user comments from a Greek sports news portal (called Gazzetta), which we make publicly available.4 This is one of the largest publicly available datasets of mod- erated user comments. We also provide word em- beddings pre-trained on 5.2M comments from the same portal. Furthermore, we experiment on the ‘attacks’ dataset of Wulczyn et al. (2017), approx. 115K English Wikipedia talk page comments la- beled as containing personal attacks or not. In a fully automatic scenario, there is no moder- ator and a system accepts or rejects comments. Al- though this scenario may be the only available one, …

Content truncated to 500 words