Duluth at SemEval-2020 Task 12: Offensive Tweet Identification in English with Logistic Regression

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1938–1946 Barcelona, Spain (Online), December 12, 2020. 1938 Duluth at SemEval-2020 Task 12: Offensive Tweet Identification in English with Logistic Regression Ted Pedersen Department of Computer Science University of Minnesota Duluth, MN 55812 USA tpederse@d.umn.edu Abstract This paper describes the Duluth systems that participated in SemEval–2020 Task 12, Multilingual Offensive Language Identification in Social Media (OffensEval–2020). We participated in the three English language tasks. Our systems provide a simple Machine Learning baseline using logistic regression. We trained our models on the distantly supervised training data made available by the task organizers and used no other resources. As might be expected we did not rank highly in the comparative evaluation: 79th of 85 in Task A, 34th of 43 in Task B, and 24th of 39 in Task C. We carried out a qualitative analysis of our results and found that the class labels in the gold standard data are somewhat noisy. We hypothesize that the extremely high accuracy (> 90%) of the top ranked systems may reflect methods that learn the training data very well but may not generalize to the task of identifying offensive language in English. This analysis includes examples of tweets that despite being mildly redacted are still offensive. 1 Introduction The goal of the OffensEval–2020 task (Zampieri et al., 2020) is to identify offensive language in tweets, and to determine if specific individuals or groups are being targeted. We relied on traditional Machine Learning methods implemented in Scikit (Pedregosa et al., 2011) to build logistic regression classifiers from distantly supervised training examples of offensive tweets (Rosenthal et al., 2020). Our methods are well known and so will only be described briefly. Instead, our primary focus in this paper is on a post-evaluation qualitative analysis of both our results and the underlying task data. Identifying offensive, abusive, and hateful language is a challenging problem that is drawing increasing attention both among the general public and in the research community (e.g.,(Fortuna and Nunes, 2018; Schmidt and Wiegand, 2017)). These are difficult problems since what is offensive depends not only on the words being used but also on the situation in which they occur. Whether something is offensive may depend on answers to questions like : Is the source of such language in a position of power? Is the target a member of a marginalized group? Is there a difference in age, race, religion, or social status of the source and the target? The number of real-world factors that may determine if language is offensive is impossible to enumerate, and leads to a high degree of polysemy in candidate offensive words that can only be unraveled by considering the social situation in which they are used. For example, words that are usually considered slurs may be used within a marginalized group as a means of bonding or identifying. The same is true of profanity, which when used within an in-group or among friends may not be offensive, while the …

Content truncated to 500 words