NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 712–721 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 712 NLPR@SRPOL at SemEval-2019 Task 6: Linguistically enhanced deep learning offensive sentence classifier Alessandro Seganti1, Helena Sobol1, Iryna Orlova1, Hannam Kim2, Jakub Staniszewski1, Tymoteusz Krumholc1, Krystian Koziel1 1 Samsung R&D Institute Poland 2 Samsung Electronics, Korea a.seganti@samsung.com Abstract The paper presents a system developed for the SemEval-2019 competition Task 5 hat- Eval Basile et al. (2019) (team name: LU Team) and Task 6 OffensEval Zampieri et al. (2019b) (team name: NLPR@SRPOL), where we achieved 2nd position in Subtask C. The system combines in an ensemble several mod- els (LSTM, Transformer, OpenAI’s GPT, Ran- dom forest, SVM) with various embeddings (custom, ELMo, fastText, Universal Encoder) together with additional linguistic features (number of blacklisted words, special charac- ters, etc.). The system works with a multi-tier blacklist and a large corpus of crawled data, annotated for general offensiveness. In the pa- per we do an extensive analysis of our results and show how the combination of features and embedding affect the performance of the mod- els. 1 Introduction In 2017 two-thirds of all adults in the United States have experienced some form of online harassment (Duggan, 2017).1 This, together with various episodes of online harassment, boosted research on the general problem of recognizing and/or fil- tering offensive language on the Internet. Still, recognizing if a sentence expresses hate speech against immigrants or women, understanding if a sentence is offensive to a group of people, an indi- vidual or others – these tasks continue to be very difficult for neural networks and machine learn- ing models to accomplish. In order to do this, various implementations have been proposed; for the most successful recent approaches see Pitsilis et al. (2018); Founta et al. (2018); Wulczyn et al. 1Due to the topic of the SemEval-2019 Tasks 5 and 6, the present paper contains offensive expressions spelled out in full. These are solely illustrations of the problems under consideration. They should not be interpreted as expressing our views in any way. (2017); Waseem and Hovy (2016); Park and Fung (2017); Davidson et al. (2017). Most of them use various combination of features to recognize these characteristics. This article presents a system that we have im- plemented for recognizing if a sentence is offen- sive. The system was developed for two SemEval- 2019 competition tasks: Task 5 hatEval “Multilin- gual detection of hate speech against immigrants and women in Twitter” Basile et al. (2019) (team name: LU Team) and Task 6 OffensEval “Identi- fying and categorizing offensive language in so- cial media” (Zampieri et al., 2019b) (team name: NLPR@SRPOL). Table 1 shows the results that we achieved with our system in the SemEval-2019 competitions. Competition Placement Task 6-A 8th position Task 6-B 9th position Task 6-C 2nd position Task 5-A 8th position (ex aequo) Table 1: SemEval-2019 results. In order to create a highly accurate classifier, we combined state-of-the-art AI with linguistic findings on the …

Content truncated to 500 words