CoLi at UdS at SemEval-2020 Task 12: Offensive Tweet Detection with Ensembling

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1916–1924 Barcelona, Spain (Online), December 12, 2020. 1916 CoLi at UdS at SemEval-2020 Task 12: Offensive Tweet Detection with Ensembling Kathryn Chapman, Johannes Bernhard, Dietrich Klakow Department of Computational Linguistics & Phonetics, Saarland University, Germany {kathrync, johannes}@coli.uni-saarland.de, dietrich.klakow@lsv.uni-saarland.de Abstract With today’s proliferation of maliciously intended communication across all social media plat- forms, finding ways of effectively combating these messages grows increasingly important. We present our submission and results for SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) where we participated in offensive tweet classi- fication tasks in English, Arabic, Greek, Turkish and Danish. Our approach included classical machine learning architectures such as support vector machines and logistic regression combined in an ensemble with a multilingual transformer-based model (XLM-R). The transformer model is trained on all languages combined in order to create a fully multilingual model which can leverage knowledge between languages. The machine learning model hyperparameters are fine-tuned and the statistically best performing ones included in the final ensemble. We further discuss the results of our model and see that our broad approach provides competitive but not task-winning performance. We also include an error analysis and potential improvements for future work. 1 Introduction In the past years, offensive language and other forms of maliciously intended online communication (trolling, cyber-bullying, fake news) have grown drastically on social media platforms such as Facebook, Twitter or YouTube. It is in the interest of everyone to inhibit this form of communication as quickly as possible once it happens. A very efficient and effective way to achieve this is via machine learning and deep learning architectures that automatically identify a message, comment or tweet as maliciously intended. These architectures have been steadily improving at this task. The purpose of the competition in SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) (Zampieri et al., 2020) is to encourage further progress in the above mentioned challenges. The organizers presented participants with tweets in English, Arabic, Greek, Turkish and Danish. For all languages, task A consisted of identifying whether a tweet was labeled as offensive (“OFF”) or non-offensive (“NOT”). Tasks B and C were only available for the English data. They consisted of identifying whether a tweet was targeted (”TIN”) or untargeted (“UNT”) for task B and in task C whether a targeted tweet was aimed at an individual (“IND”), group (“GRP”) or other (“OTH”). We participated in task A for all languages and task B for English. Data set sizes and label balance varied greatly between languages which provided additional challenges when deciding on how to approach the task. We chose to construct an ensemble of more traditional machine learning architectures (SVM, Logistic Regression, etc.) together with a multilingual transformer-based model (XLM-R) (Conneau et al., 2019) with the thought that the strengths of the different models would help the ensemble generate the most well-rounded results. The ensemble generated promising results during experiments and was submitted for all languages in …

Content truncated to 500 words