NULI at SemEval-2019 Task 6: Transfer Learning for Offensive Language Detection using Bidirectional Transformers

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 87–91 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 87 NULI at SemEval-2019 Task 6: Transfer Learning for Offensive Language Detection using Bidirectional Transformers Ping Liu Department of Computer Science Illinois Institute of Technology pliu19@hawk.iit.edu Wen Li Department of Linguistics Indiana University wl9@indiana.edu Liang Zou Department of Mathematics New York University lz1904@nyu.edu Abstract Transfer learning and domain adaptive learn- ing have been applied to various fields in- cluding computer vision (e.g., image recog- nition) and natural language processing (e.g., text classification). One of the benefits of transfer learning is to learn effectively and ef- ficiently from limited labeled data with a pre- trained model. In the shared task of identi- fying and categorizing offensive language in social media, we preprocess the dataset ac- cording to the language behaviors on social media, and then adapt and fine-tune the Bidi- rectional Encoder Representation from Trans- former (BERT) pre-trained by Google AI Lan- guage team1. Our team NULI wins the first place (1st) in Sub-task A - Offensive Lan- guage Identification and is ranked 4th and 18th in Sub-task B - Automatic Categorization of Offense Types and Sub-task C - Offense Tar- get Identification respectively. 1 Introduction Anti-social online behaviors, including cyberbul- lying, trolling and offensive language (Xu et al., 2012; Kwok and Wang, 2013; Cheng et al., 2017), are attracting more attention on different social networks. The intervention of such behaviors should be taken at the earliest opportunity. Auto- matic offensive language detection using machine learning algorithms becomes one solution to iden- tifying such hostility and has shown promising performance. In SemEval-2019 Task 6: Identifying and Cat- egorizing Offensive Language in Social Media (Zampieri et al., 2019b), the organizers collected tweets through Twitter API and annotated them hi- erarchically regarding offensive language, offense type, and offense target. The task is divided into three sub-tasks: a) detecting if a post is offensive 1https://github.com/google-research/bert (OFF) or not (NOT); b) identifying the offense type of an offensive post as targeted insult (TIN), targeted threat (TTH), or untargeted (UNT); c) for a post labeled as TIN/TTH in sub-task B, identify- ing the target of offense as individual (IND), group of people (GRP), organization or entity (ORG), or other (OTH). The three sub-tasks are indepen- dently evaluated by macro-F1 metric. The challenges of this shared task include: a) comparatively small dataset makes it hard to train complex models; b) the characteristics of language on social media pose difficulties such as out-of- vocabulary words and ungrammatical sentences; c) the distribution of target classes is imbalanced and inconsistent between training and test data. To address the problem of out-of-vocabulary words especially emoji and hashtags, we preprocess each tweet by interpreting emoji as meaningful En- glish phrases and segmenting hashtags into space separated words. The classifiers we experiment with include: linear model with features of word unigrams, word2vec, and Hatebase; word-based Long Short-Term Memory (LSTM); fine-tuned Bidirectional Encoder Representation from Trans- former (BERT) (Devlin et al., …

Content truncated to 500 words