DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1932–1937 Barcelona, Spain (Online), December 12, 2020. 1932 DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection Zoher Orabe*, Bushr Haddad*, Anas Al-Abood*, Nada Ghneim** **Damascus University / Damascus, Syria ***AlSham Private University / Damascus, Syria {zoherorabe999, bushr.haddad, anasabood3}@gmail.com n.ghneim@aspu.edu.com Abstract This paper describes our team work and submission for the SemEval 2020 (Sub-Task A) “Offensive Eval: Identifying and Categorizing Offensive Arabic Language in Arabic Social Media”. Our two baseline models were based on different levels of representation: character vs. word level. In word level based representation we implemented a convolutional neural network model and a bi-directional GRU model. In character level based representation we implemented a hyper CNN and LSTM model. All of these models have been further augmented with attention layers for a better performance on our task. We also experimented with three types of static word embeddings: word2vec, FastText, and Glove, in addition to emoji embeddings, and compared the performance of the different deep learning models on the dataset provided by this task. The bi-directional GRU model with attention has achieved the highest score (0.85% F1 score) among all other models. 1 Introduction With the massive increase of social interactions on online social networks, and the expansion of opportu- nities provided by these networks, there has also been a rise of offensive and hateful content that exploits and violates such networks. Offensive language is defined as any explicit or implicit textual insult or attack against other people. Offensive language has many variants such as hate speech, which is defined as any targeted offensive language toward a particular person (a politician, a celebrity, etc.) or a particular group (a gender, a religion, a color). Offensive language is considered a critical problem to the social networks for its huge psychological negative impact on users. This work has been prepared and organized as a participation at OffensEval2020 competition (Zampieri et al., 2020). Zampieri (2020) features a multilingual dataset with five languages (Arabic, Danish, English, Greek, and Turkish). We participated in the sub-task dedicated for Arabic language namely sub-task A (Arabic offensive language detection) (Mubarak et al., 2020). In this work, we present our deep learning models featured on word and character level and aimed to identify offensive language in Arabic tweets. In the rest of this paper, a brief of related works are presented in section 2. In section 3, we present our data preparation, and representation. Our proposed models are developed in section 4. In section 5, a brief discussion on the results is addressed. At the end, a short summary and some future insights are presented. 2 Related Works Several works have addressed the problem of offensive language detection and its variants (Cambray and Podsadowski, 2019; Zhang and Luo, 2019; Gamb¨ack and Sikdar, 2017; Badjatiya et al., 2017). Cambray (2019) competed on OffensEval 2019, and presented their best model as a bidirectional LSTM; followed by a two-branch bidirectional LSTM and …

Content truncated to 500 words