Smatgrisene at SemEval-2020 Task 12: Offense Detection by AI - with a Pinch of Real I

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2140–2145 Barcelona, Spain (Online), December 12, 2020. 2140 Smatgrisene at SemEval-2020 Task 12: Offense detection by AI – with a pinch of real I Peter Juel Henrichsen Marianne Rathje Danish Language Council Danish Language Council pjh@dsn.dk mr@dsn.dk Abstract This paper discusses how ML based classifiers can be enhanced disproportionately by adding small amounts of qualitative linguistic knowledge. As an example we present the Danish classifier Smatgrisene, our contribution to the recent OffensEval Challenge 2020. The classifier was trained on 3000 social media posts annotated for offensiveness, supplemented by rules extracted from the reference work on Danish offensive language (Rathje 2014b). Smatgrisene did surprisingly well in the competition in spite of its extremely simple design, showing an interesting trade-off between technological muscle and linguistic intelligence. Finally, we comment on the perspectives in combining qualitative and quantitative methods for NLP.1 1 Introduction Offense in the social media (SoMe) is a nuisance, not only for the targeted victim, but for editors, consumers, and stakeholders - even damaging the reputation of the internet as a whole. Could we just detect and eliminate offensive SoMe posts with the same security as we can syntax errors in python programs or even in human texts, much would be gained. The need for automatic offense detection motivated the recent OffensEval Challenge 2020 (Sigurbergsson et al. 2020, https://arxiv.org/pdf/2006.07235.pdf). In this paper we present our Danish classifier (called Smatgrisene) and report on our participation in OffensEval. After introducing OffensEval we discuss the annotation principles behind the shared training materials and comment on a possible consistency problem. We suggest, as a remedy, a classifier design based on a mixture of simple surface rules and synthetic linguistic knowledge. We then present our OffensEval results, and conclude with a discussion of knowledge-enhanced NLP for offense detection and beyond. 2 OffensEval 2020 - Danish chapter The Danish training material consisted of 3290 SoMe posts each annotated with labels OFF ('offensive') or NOT ('not offensive'). index text label #443 jeg spørger lige dumt: hvad er det for et spil? NOT #507 Var det ham der sad i en lort? OFF #1327 Rigtig god bedring til vores Dronning ... NOT #2504 Holy fuck, fik lige en orgasme takket være dette billede. OFF Figure 1. Samples from the OffensEval training material ("asking a stupid question: what game is it?", "was it him sitting in a shit?", "Wishing a quick recovery for our queen", "Holy fuck, I just had an orgasm from this picture") 1 This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http://creativecommons.org/licenses/by/4.0/. 2141 Parts of the annotations were kept secret from the participants (the test corpus) while the largest part was available as training data (2961 posts). Each participating team trained an offense classifier, applied it to the test corpus, and submitted the output as their contribution. The Danish chapter of OffensEval 2020 had 39 contributing teams including Smatgrisene, the authors of this paper. 2.1 The meaning of OFF As …

Content truncated to 500 words