SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1425–1447 Barcelona, Spain (Online), December 12, 2020. 1425 SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) Marcos Zampieri1, Preslav Nakov2, Sara Rosenthal3, Pepa Atanasova4, Georgi Karadzhov5 Hamdy Mubarak2, Leon Derczynski6, Zeses Pitenis7, C¸ a˘grı C¸ ¨oltekin8 1Rochester Institute of Technology, USA, 2Qatar Computing Research Institute, Qatar 3IBM Research, USA, 4University of Copenhagen, Denmark, 5University of Cambridge, UK 6IT University Copenhagen, Denmark, 7University of Wolverhampton, UK 8University of T¨ubingen, Germany marcos.zampieri@rit.edu Abstract We present the results and the main findings of SemEval-2020 Task 12 on Multilingual Offensive Language Identification in Social Media (OffensEval-2020). The task included three subtasks corresponding to the hierarchical taxonomy of the OLID schema from OffensEval-2019, and it was offered in five languages: Arabic, Danish, English, Greek, and Turkish. OffensEval-2020 was one of the most popular tasks at SemEval-2020, attracting a large number of participants across all subtasks and languages: a total of 528 teams signed up to participate in the task, 145 teams submitted official runs on the test data, and 70 teams submitted system description papers. 1 Introduction Offensive language is ubiquitous in social media platforms such as Facebook, Twitter, and Reddit, and it comes in many forms. Given the multitude of terms and definitions related to offensive language used in the literature, several recent studies have investigated the common aspects of different abusive language detection tasks (Waseem et al., 2017; Wiegand et al., 2018). One such example is SemEval-2019 Task 6: OffensEval1(Zampieri et al., 2019b), which is the precursor to the present shared task. OffensEval- 2019 used the Offensive Language Identification Dataset (OLID), which contains over 14,000 English tweets annotated using a hierarchical three-level annotation schema that takes both the target and the type of offensive content into account (Zampieri et al., 2019a). The assumption behind this annotation schema is that the target of offensive messages is an important variable that allows us to discriminate between, e.g., hate speech, which often consists of insults targeted toward a group, and cyberbullying, which typically targets individuals. A number of recently organized related shared tasks followed similar hierarchical models. Examples include HASOC-2019 (Mandl et al., 2019) for English, German, and Hindi, HatEval-2019 (Basile et al., 2019) for English and Spanish, GermEval-2019 for German (Struß et al., 2019), and TRAC-2020 (Kumar et al., 2020) for English, Bengali, and Hindi. OffensEval-2019 attracted nearly 800 team registrations and received 115 official submissions, which demonstrates the interest of the research community in this topic. Therefore, we organized a follow-up, OffensEval-20202 (SemEval-2020 Task 12), which is described in this report, building on the success of OffensEval-2019 with several improvements. In particular, we used the same three-level taxonomy to annotate new datasets in five languages, where each level in this taxonomy corresponds to a subtask in the competition: • Subtask A: Offensive language identification; • Subtask B: Automatic categorization of offense types; • Subtask C: Offense target identification. This work is licensed under a Creative Commons Attribution 4.0 International License. …

Content truncated to 500 words