Lee at SemEval-2020 Task 12: A BERT Model Based on the Maximum Self-ensemble Strategy for Identifying Offensive Language

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2067–2072 Barcelona, Spain (Online), December 12, 2020. 2067 Lee at SemEval-2020 Task 12: A BERT model based on the maximum self-ensemble strategy for identifying offensive language Junyi Li, Xiaobing Zhou∗, Zichen Zhang School of Information Science and Engineering Yunnan University, Kunming 65091, P.R. China ∗Corresponding author, zhouxb@ynu.edu.cn Abstract This article describes the system submitted to SemEval 2020 Task 12: OffensEval 2020. This task aims to identify and classify offensive languages in different languages on social media. We only participate in the English part of subtask A, which aims to identify offensive languages in English. To solve this task, we propose a BERT model system based on the transform mechanism, and use the maximum self-ensemble to improve model performance. Our model achieved a macro F1 score of 0.913(ranked 13/82) in subtask A. 1 Introduction In recent years, due to the rapid development of mobile internet and social media platforms, people have begun to use a variety of social media to share their lives, such as Facebook and Twitter. People share their something like events on social media, and this behavior can receive good or bad comments. Some bad reviews slowly evolve into offensive language. A large amount of offensive language is flooded in social media, and filtering out offensive language has become an important thing. Because manual screening is very time consuming and can cause symptoms such as post-traumatic stress disorder, many studies have focused on automating this process. Task 12 of Semeval 2020 (Zampieri et al., 2020) is the second edition of task 6 of Semeval 2019 (Zampieri et al., 2019b). The purpose of this task is to identify online offensive language . Its goal is to use computational methods to identify offensive and hate speech in user-generated content on online social media platforms. We can use this method to prevent the abuse of offensive words in social media. This task gives us some social media platforms and categorizes the content In this task, we only participate in subtask A (Rosenthal et al., 2020): Recognizing offensive language. For this task, we use deep learning methods. We mainly use BERT model based on Transformer mechanism and model ensemble. After data processing, we input the generated word vector into the pre-trained model. In addition, we also adopted the method of model ensemble to optimize the prediction probability of the results in order to get better results. Facts have proved that this approach is an effective method. The rest of our paper is structured as follows. Section 2 describes data preparation. Models are described in Section 3. Experiments and evaluation are described in Section 4. The conclusions are drawn in Section 5. 2 Data Preparation The organizers provided training and test sets (Mubarak et al., 2020), containing 9073396 and 3887 sentences respectively. We note that the data set format of task 12 is different from that of the previous year (Zampieri et al., 2019a). Therefore, before we perform data preprocessing, we need to …

Content truncated to 500 words