Examining the Effects of Preprocessing on the Detection of Offensive Language in German Tweets

Examining the Effects of Preprocessing on the Detection of Offensive Language in German Tweets Sebastian Reimann Uppsala University Department of Linguistics reimann.sebastian9483@gmail.com Daniel Dakota Uppsala University Department of Linguistics ddakota@lingfil.uu.se Abstract Preprocessing is essential for creating more ef- fective features and reducing noise in classifi- cation, especially in user-generated data (e.g. Twitter). How each individual preprocessing decision changes an individual classifier’s be- havior is not universal. We perform a series of ablation experiments in which we examine how classifiers behave based on individual pre- processing steps when detecting offensive lan- guage in German. While preprocessing deci- sions for traditional classifier approaches are not as varied, we note that pre-trained BERT models are far more sensitive to each decision and do not behave identically to each other. We find that the cause of much variation be- tween classifiers has to do with the interactions specific preprocessing steps have on the over- all vocabulary distributions, and, in the case of BERT models, how this interacts with the WordPiece tokenization. 1 Introduction The task of abusive language detection has be- come increasingly popular for a variety of lan- guages (Zampieri et al., 2019; Basile et al., 2019; Al-Khalifa et al., 2020). German specifically has had two shared tasks on the topic, one in 2018 (Wiegand et al., 2018) and a second in 2019 (Struß et al., 2019). Not only is offensive language detection some- what subjective in nature, particularly in the need for contextual requirements, but is often examined through user generated mediums, creating another layer of complexity to successfully identify possi- ble abusive language. Often, in order to create more useful features out of the text for the classifier, we must first treat the text to reduce the noise. While for standard feature generation via count vectors the impact is far more obvious (e.g. reduction of feature space), even when we feed a dense vector representations to a classifier (e.g. a sentence em- bedding), that embedding still represents the textual representation, simply in an alternative way. Thus, it too is influenced by individual alterations to the text. With languages that show large variation in di- alect preferences and orthographic representations, this has been shown to be particularly important (Husain, 2020). Twitter has proven to be a typical source for not only research on offensive language, but also neces- sitating additional preprocessing approaches given its different style of communication and lexicon. In this work we look to perform a set of ablation experiments in which we evaluate how different preprocessing techniques impact classifier behav- ior over three different approaches to classification when detecting offensive language in German Twit- ter. We seek to answer the following questions: 1. How do different preprocessing techniques influence performance across different classi- fiers? 2. Can we identify features within different pre- proccesing techniques that can help explain specific classifier behaviors? 2 Related Work 2.1 Data Ross et al. (2016) introduced a Twitter corpus of offensive language detection, examining the 2015 refugee crisis. They predominantly focused on user’s perceptions …

Content truncated to 500 words