UoT-UWF-PartAI at SemEval-2021 Task 5: Self Attention Based Bi-GRU with Multi-Embedding Representation for Toxicity Highlighter

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 948–952 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 948 UoT-UWF-PartAI at SemEval-2021 Task 5: Self Attention Based Bi-GRU with Multi-Embedding Representation for Toxicity Highlighter Hamed Babaei Giglou Department of Computer Science University of Tabriz, Tabriz, Iran h.babaei98@ms.tabrizu.ac.ir Taher Rahgooy Department of Computer Science University of West Florida, Florida, USA trahgooy@students.uwf.edu Mostafa Rahgouy Part AI Research Center Tehran, Iran mostafa.rahgouy@partdp.ai Jafar Razmara Department of Computer Science University of Tabriz, Tabriz, Iran razmara@tabrizu.ac.ir Abstract Toxic Spans Detection(TSD) task is defined as highlighting spans that make a text toxic. Many works have been done to classify a given comment or document as toxic or non-toxic. However, none of those proposed models work at the token level. In this paper, we propose a self-attention-based bidirectional gated recur- rent unit(BiGRU) with a multi-embedding rep- resentation of the tokens. Our proposed model enriches the representation by a combination of GPT-2, GloVe, and RoBERTa embeddings, which led to promising results. Experimen- tal results show that our proposed approach is very effective in detecting span tokens. 1 Introduction With the massive increase in social interactions on online social networks, keeping discussions fruitful is a central concern for platform providers. Indeed, abusive (e.g., bullying, profanity, hate speech), damaging the reputation of a platform. Thus, it is necessary to be detected by automated machine learning systems because of huge amount of data. However, the previous works mostly focus on whether the given document(e.g. comment) is toxic or not. Detecting the span tokens in the docu- ment may be more beneficial. For example, it can prevent users to use span tokens before they send their post. Also, it is useful to filter span tokens before learning AI chatterbots instead of removing the whole documents. In this paper, we propose a self-attention-based bidirectional gated recurrent unit (BiGRU) with a multi-embedding representation of the tokens. Our proposed model enriches representation showed very promising results. The rest of the paper is organized as follows. Section 2 provides background and presents some related works on TSD in general. Section 3 intro- duces our BiGRUs model. Results are covered in Section 4. In Section 5 we draw a conclusion. 2 Related Research A toxic post (comments) is defined as a rude, dis- respectful, or unreasonable comment that is likely to make other users leave a discussion. A goal of toxic comment classification is to give a right to freedom of expression on the web. Training complex neural networks (NN) requires enough datasets of toxic comments. Word em- beddings are the basis of the NNs when working with text data. NNs for toxic comment classifica- tion use Recurrent Neural Networks (RNN) layers such as Long short-term memory (LSTM) (Hochre- iter and Schmidhuber, 1997) or Gated Recurrent Unit (GRU) (Chung et al., 2014) layers. Similarly, the attention mechanism for NNs has been suc- cessfully applied to toxic comment classification (Pavlopoulos et al., 2017; van Aken et al., 2018). In semi-automated content-moderation, attention can …

Content truncated to 500 words