Duluth at SemEval-2019 Task 6: Lexical Approaches to Identify and Categorize Offensive Tweets

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 593–599 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 593 Duluth at SemEval-2019 Task 6: Lexical Approaches to Identify and Categorize Offensive Tweets Ted Pedersen Department of Computer Science University of Minnesota Duluth, MN 55812 USA tpederse@d.umn.edu Abstract This paper describes the Duluth systems that participated in SemEval–2019 Task 6, Identi- fying and Categorizing Offensive Language in Social Media (OffensEval). For the most part these systems took traditional Machine Learn- ing approaches that built classifiers from lexi- cal features found in manually labeled training data. However, our most successful system for classifying a tweet as offensive (or not) was a rule-based black–list approach, and we also experimented with combining the training data from two different but related SemEval tasks. Our best systems in each of the three OffensE- val tasks placed in the middle of the compara- tive evaluation, ranking 57th of 103 in task A, 39th of 75 in task B, and 44th of 65 in task C. 1 Introduction Social media is notorious for providing a platform for offensive, toxic, and hateful speech. There is a pressing need for NLP tools that can identify and moderate this type of content. The OffensEval task (Zampieri et al., 2019b) fo- cuses on identifying offensive language in tweets, and determining if specific individuals or groups are being targeted. Our approach was to rely on traditional Machine Learning methods as im- plemented by Scikit (Pedregosa et al., 2011) to build classifiers from manually labeled examples of offensive tweets. Our models relied on lexical features, including character ngrams, words, and word ngrams. We also included a dictionary based black–list approach, and experimented with com- bining training data from two related yet different SemEval-2019 tasks. Offensive language is an umbrella term that covers hate speech, cyber-bullying, abusive lan- guage, profanity, and so on. Recognizing offen- sive language is an important first step in deal- ing with different kinds of problematic text. Lan- guage that is offensive may simply violate com- munity standards regarding the use of profanity, but in other cases may cross over to become abu- sive, threatening, or dangerous. Detecting such language has proven to be a challenging prob- lem, at least in part because it remains diffi- cult to make distinctions between the casual and even friendly use of profanity versus more serious forms of offensive language (Fortuna and Nunes, 2018; Schmidt and Wiegand, 2017). 2 Experimental Data OffensEval is made up of three tasks that were carried out in stages during January 2019. Task A is to classify a tweet as being offensive (OFF) or not (NOT). Task B takes the offensive tweets from task A and decides if they are targeted insults (TIN) or not (UNT). Task C looks at the targeted insults from task B and classifies them as being directed against an individual (IND), group (GRP) or other entity (OTH). Task A provides 13,240 training tweets, of which 8,840 (66.7%) were not offensive …

Content truncated to 500 words