UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection on Social Media by Fine-tuning a Variety of BERT-based Models

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 2222–2231 Barcelona, Spain (Online), December 12, 2020. 2222 UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection on Social Media by Fine-tuning a Variety of BERT-based Models Mircea-Adrian Tanase, Dumitru-Clementin Cercel, Costin-Gabriel Chiru University Politehnica of Bucharest, Faculty of Automatic Control and Computers mircea.tanase@stud.acs.upb.ro {dumitru.cercel, costin.chiru}@cs.pub.ro Abstract Offensive language detection is one of the most challenging problem in the natural language pro- cessing field, being imposed by the rising presence of this phenomenon in online social media. This paper describes our Transformer-based solutions for identifying offensive language on Twit- ter in five languages (i.e., English, Arabic, Danish, Greek, and Turkish), which was employed in Subtask A of the Offenseval 2020 shared task. Several neural architectures (i.e., BERT, mBERT, Roberta, XLM-Roberta, and ALBERT), pre-trained using both single-language and multilingual corpora, were fine-tuned and compared using multiple combinations of datasets. Finally, the highest-scoring models were used for our submissions in the competition, which ranked our team 21st of 85, 28th of 53, 19th of 39, 16th of 37, and 10th of 46 for English, Arabic, Danish, Greek, and Turkish, respectively. 1 Introduction Social media platforms are gaining increasing popularity for both personal and political communication. Recent studies uncovered disturbing trends in communications on the Internet. For example, Pew Re- search Center1 discovered that 60% of Internet users have witnessed a form of online harassment, while 41% of them have personally experienced it. The majority of the latter category says the most recent such experience occurred on a social media platform. Although most of these platforms provide ways of reporting offensive or hateful content, only 9% of the victims have considered using these tools. Traditionally, identifying and removing offensive or hateful content on the Internet is performed by human moderators that inspect each piece of content flagged by the users and label it appropriately. This process has two major disadvantages. As previously mentioned, the first one is that the proportion of users that even considered using the tools provided by the platforms in order to report the harmful content is very small. The second one is represented by the continuously growing volume of data that needs to be analyzed. However, the task of automated offensive language detection on social media is a very complicated problem. This is because the process of labeling an offensive language dataset has proved to be very challenging, as every individual reacts differently to the same content, and the consensus on assigning a label to a piece of content is often difficult to obtain (Waseem et al., 2017). The SemEval-2019 shared task 6 (Zampieri et al., 2019b), Offenseval 2019, was the first competition oriented towards detecting offensive language in social media, specifically Twitter. The SemEval-2020 shared task 12 (Zampieri et al., 2020), Offenseval 2020, proposes the same problem, the novelty being a very large automatically labeled English dataset and also smaller datasets for four other languages: Ara- bic, Danish, Greek, and Turkish. This paper describes the Transformer-based machine learning …

Content truncated to 500 words