BNU-HKBU UIC NLP Team 2 at SemEval-2019 Task 6: Detecting Offensive Language Using BERT model

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 551–555 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 551 BNU-HKBU UIC NLP Team 2 at SemEval-2019 Task 6: Detecting Offensive Language Using BERT model Zhenghao Wu Hao Zheng Jianming Wang Weifeng Su Jefferson Fong {l630003054,l630003067,l630003049}@mail.uic.edu.hk {wfsu,jeffersonfong}@uic.edu.hk Computer Science and Technology, Division of Science and Technology BNU-HKBU United International College Zhuhai, Guangdong, China Abstract In this study we deal with the problem of iden- tifying and categorizing offensive language in social media. Our group, BNU-HKBU UIC NLP Team2, use supervised classification along with multiple version of data generated by different ways of pre-processing the data. We then use the state-of-the-art model Bidirec- tional Encoder Representations from Trans- formers, or BERT (Devlin et al. (2018)), to capture linguistic, syntactic and semantic fea- tures. Long range dependencies between each part of a sentence can be captured by BERT’s bidirectional encoder representations. Our results show 85.12% accuracy and 80.57% F1 scores in Subtask A (offensive language identification), 87.92% accuracy and 50% F1 scores in Subtask B (categorization of offense types), and 69.95% accuracy and 50.47% F1 score in Subtask C (offense target identifica- tion). Analysis of the results shows that distin- guishing between targeted and untargeted of- fensive language is not a simple task. More work needs to be done on the unbalance data problem in Subtasks B and C. Some future work is also discussed. 1 Introduction Social media is an essential part of human com- munication today. People can share their opinions in this platform with anonymity. Some people use offensive language and hate speech casually and frequently without taking any responsibility for their behavior. For this reason, SemEval 2019 (Zampieri et al. (2019b)) set up the task Offen- sEval: identifying and categorizing offensive language in social media. This task is divided into three subtasks: offensive language identification, automatic categorization of offensive types, and offence target identification. Our group uses the Natural Language Process- ing (NLP) latest model, Bidirectional Encoder Representations from Transformers (BERT). It is a general-purpose “language understanding” model trained on a large text corpus such as Wikipedia (Devlin et al. (2018)). After fine- tuning, the model can be used for downstream NLP tasks. Because BERT is very complex and is the state-of-art model, it is prudent for us not to change its internal structure. Hence, we focus on preprocessing the data and error analysis. After much experimentation with the data, such as translating emoji into words, putting more weight on some metaphorical words, removing the hashtag and so on, we find that using the original data will give the best performance. The reason for this is perhaps if we remove some information from the sentence, some features that affect the prediction result will be lost. So we end up using the original data to train our model. 2 Related Work Much research has been done in detecting of- fensive language, aggression, and hate speech in user-generated content. In recent years, re- searches tend to …

Content truncated to 500 words