UBC-NLP at SemEval-2019 Task 6: Ensemble Learning of Offensive Content With Enhanced Training Data

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 775–781 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 775 UBC-NLP at SemEval-2019 Task 6: Ensemble Learning of Offensive Content With Enhanced Training Data Arun Rajendran Chiyu Zhang Muhammad Abdul-Mageed Natural Language Processing Lab University of British Columbia muhammad.mageed@ubc.ca Abstract We examine learning offensive content on Twitter with limited, imbalanced data. For the purpose, we investigate the utility of using var- ious data enhancement methods with a host of classical ensemble classifiers. Among the 75 participating teams in SemEval-2019 sub-task B, our system ranks 6th (with 0.706 macro F1- score). For sub-task C, among the 65 partici- pating teams, our system ranks 9th (with 0.587 macro F1-score). 1 Introduction With the proliferation of social media, millions of people currently express their opinions freely on- line. Unfortunately, this is not without costs as some users fail to maintain the thin line between freedom of expression and hate speech, defama- tion, ad hominem attacks, etc. Manually detecting these types of negative content is not feasible, due to the sheer volume of online communication. In addition, individuals tasked with inspecting such types of content may suffer from depression and burnout. For these reasons, it is desirable to build machine learning systems that can flag offensive online content. Several works have investigated detecting un- desirable (Alshehri et al., 2018) and offensive language online using traditional machine learn- ing methods. For example, Xiang et al. (2012) employ statistical topic modelling and feature en- gineering to detect offensive tweets. Similarly, Davidson et al. (2017) train multiple classifiers (e.g., logistic regression, decision trees, and sup- port vector machines) to detect hate speech from general offensive tweets. More recently, deep ar- tificial neural networks (i.e., deep learning) has been used for several text classification tasks, in- cluding detecting offensive and hateful language. For example, Pitsilis et al. (2018) use recurrent neural networks (RNN) to detect offensive lan- guage in tweets. Mathur et al. (2018) use trans- fer learning with convolutional neural networks (CNN) for offensive tweet classification on Twit- ter data. Most of these works, however, either assume relatively balanced data (traditional classifiers) and/or large amounts of labeled data (deep learn- ing). In scenarios where only highly imbalanced data are available, it becomes challenging to learn good generalizations. In these cases, it is useful to employ methods with good predictive power for especially minority classes. For example, meth- ods capable of enhancing training data (e.g., by augmenting minority categories) are desirable in such scenarios. In the literature, some works have been undertaken to address issues of data imbal- ance in language tasks. For example, Moun- tassir et al. (2012) propose different undersam- pling techniques that yield better performance than common random undersampling on senti- ment analysis. Along similar lines Gopalakrish- nan and Ramaswamy (2014) propose a modified ensemble based bagging algorithm and sampling techniques that improve sentiment analysis. Fur- ther, Li et al. (2018) present a novel oversampling technique that generates synthetic texts from …

Content truncated to 500 words