LT@Helsinki at SemEval-2020 Task 12: Multilingual or Language-specific BERT?

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1569–1575 Barcelona, Spain (Online), December 12, 2020. 1569 LT@Helsinki at SemEval-2020 Task 12: Multilingual or language-specific BERT? Marc P`amies Emily ¨Ohman University of Helsinki firstname.lastname@helsinki.fi Kaisla Kajava J¨org Tiedemann Abstract This paper presents the different models submitted by the LT@Helsinki team for the SemEval 2020 Shared Task 12. Our team participated in sub-tasks A and C; titled offensive language identification and offense target identification, respectively. In both cases we used the so-called Bidirectional Encoder Representation from Transformer (BERT), a model pre-trained by Google and fine-tuned by us on the OLID and SOLID datasets. The results show that offensive tweet classification is one of several language-based tasks where BERT can achieve state-of-the-art results. 1 Introduction The number of social media users has reached 3.5 billion, and an average of 6,000 tweets are generated every second (Mathew et al., 2019) . With such a large volume of tweets, it seems inevitable that some use offensive language. A study from 2014 found that 67% of social media users had been exposed to online hate, and 21% had been the target of online hate (Oksanen et al., 2014). The usual approach on social media sites is to forbid hate speech in the terms of service and to censor any inappropriate content detected by their algorithms, but still, companies like Facebook or Twitter have been harshly criticized for not doing enough (Del Vigna et al., 2017). This criticism has forced companies to try and find accurate and scalable solutions that solve the problem of offensive language detection using automated methods. Workshops dealing with offensive language, such as TRAC (Kumar et al., 2018b), TA-COS (Lefever et al., 2018), or ALW1 (Waseem et al., 2017), as well as shared tasks like GermEval (Wiegand et al., 2018), TRAC-1 (Kumar et al., 2018a), and OffensEval 2019 (Zampieri et al., 2019b), are becoming more and more prevalent. The task we address here, SemEval 2020 task 12 (Zampieri et al., 2020), is titled Multilingual Offen- sive Language Identification in Social Media and is divided into the following sub-tasks: A. Offensive Language Identification in several languages (Arabic, Danish, English, Greek, Turkish): whether a tweet is offensive or not. B. Categorization of Offense Types: whether an offensive tweet is targeted or untargeted. C. Offense Target Identification: whether a targeted offensive tweet is directed towards an individual, a group or otherwise. In this paper the system created by the LT@Helsinki team for sub-tasks A and C will be described. In sub-task A we participated in all the language tracks. For sub-task C the only language available was English. We qualified as second in sub-task C and our submission for sub-task A ranked first for Danish, seventh for Greek, eighteenth for Turkish, and forty-sixth for Arabic. In all submissions we used BERT-Base models (Devlin et al., 2019) fine-tuned on each dataset provided by the task organizers. We also experimented with random forest with TF-IDF and other kinds of features, but the results on the development …

Content truncated to 500 words