Offence in Dialogues: A Corpus-Based Study

Proceedings of Recent Advances in Natural Language Processing, pages 1085–1093, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_125 1085 Offence in Dialogues: A Corpus-Based Study Johannes Sch¨afer1 and Ben Burtenshaw2 1 Institute for Information Science and Natural Language Processing University of Hildesheim, Universit¨atsplatz 1, Hildesheim, Germany johannes.schaefer@uni-hildesheim.de 2 Computational Linguistics & Psycholinguistics Research Center The University of Antwerp, Lange Winkelstraat 40-42, Antwerp, Belgium benjamin.burtenshaw@uantwerpen.be Abstract In recent years an increasing number of analyses of offensive language has been published, however, dealing mainly with the automatic detection and classification of isolated instances. In this paper we aim to understand the impact of offen- sive messages in online conversations di- achronically, and in particular the change in offensiveness of dialogue turns. In turn, we aim to measure the progression of of- fence level as well as its direction – For example, whether a conversation is esca- lating or declining in offence. We present our method of extracting linear dialogues from tree-structured conversations in so- cial media data and make our code pub- licly available.1 Furthermore, we discuss methods to analyse this dataset through changes in discourse offensiveness. Our paper includes two main contributions; first, using a neural network to measure the level of offensiveness in conversations; and second, the analysis of conversations around offensive comments using decou- pling functions. 1 Introduction Offensive language is a complex problem, espe- cially in social media where operators are required to counter illegal hate speech in user-generated content. However, it is not clear what counts as offensive language since even humans struggle to find objective definitions (Chen et al., 2012). The general approach to this problem is to train sys- tems for the detection of such unwanted content 1https://github.com/Johannes-Schaefer/ oid_ranlp19 based on human annotations of empirically gath- ered data instances. Several shared tasks engaged with the topic of offensive language detection in recent years, for example OffensEval-2019 for “Identifying and Categorizing Offensive Language in Social Me- dia” (Zampieri et al., 2019b) or GermEval-2018, the “Shared Task on the Identification of Offen- sive Language” (Wiegand et al., 2018). The dif- ficulty for machine learning systems at this task becomes apparent when considering the perfor- mance scores of the submitted systems. For exam- ple, at GermEval-2018 the best performing sub- mitted system reached a macro-averaged F1-score of only 76.77 %. Here, systems struggle to simul- taneously detect all various types of offensive lan- guage and it remains highly questionable if we can act on these automatic predictions and delete de- tected offensive language in practical applications. Following the hint that deletion based on pre- dictions of a machine learning system might not be the most appropriate course of action, we try to approach the problem of offensive language from another direction. In almost the same manner as mentioned above, we rely on machine learning of annotated instances to detect messages which might be offensive; however, we intend to act differently. Rather than deleting supposedly un- wanted instances, we suggest to use tactics to counter offence. We aim for an empirical …

Content truncated to 500 words