EmoNLP at SemEval-2018 Task 2: English Emoji Prediction with Gradient Boosting Regression Tree Method and Bidirectional LSTM

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 390–394 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics EmoNLP at SemEval-2018 Task 2: English Emoji Prediction with Gradient Boosting Regression Tree Method and Bidirectional LSTM Man Liu liumanlalala@gmail.com Abstract This paper describes our system used in the English Emoji Prediction Task 2 (Bar- bieri et al., 2018) at the SemEval-2018. Our system is based on two supervised machine learning algorithms: Gradient Boosting Regression Tree Method (GBM) and Bidirectional Long Short-term Mem- ory Network (BLSTM). Besides the com- mon features, we extract various lexicon and syntactic features from external re- sources. After comparing the results of two algorithms, GBM is chosen for the fi- nal evaluation. 1 Introduction Short text messages from social media websites such as Twitter and Facebook have become an im- portant communication channel in our daily life. Although the writing styles of such short text mes- sages are extremely diverse, the usage of emojis are generally shared. Emojis are ideograms and smiles that can be electronic expressions of natu- ral emotions. Genres of emojis vary from facial expression, places, types of weather to animals. Emojis are used every day, rapidly changing the communication way in the social network. Due to the importance of emojis, investigations about emojis have been performed in recent years. For example, the previous work (Barbieri et al., 2017) has shown that there are relations between words and emojis, in other words, emojis are predictable given its surrounding words. Task 2 of SemEval 2018 provides a platform for the further prediction of emojis on tweets. Our system addresses the first subtask: English emoji prediction. Note that all the emojis in the training data are removed. We models this prob- lem as a multiclass classification problem. Specif- ically, we leverage on semantic and syntactic re- sources to extract varieties of features. After fea- ture engineering, two models are adopted: gradi- ent boosting regression tree method (GBM) and bi-directional long short-term memory network (BLSTM). After comparing results of these two models, GBM is selected for the final evaluation. The reminder of this paper is structured as fol- lows. In section 2, we describe our system in detail, including the feature description and ap- proaches. In section 3, results of 5-fold experi- ments and feature ablation are presented. Finally, section 4 summarizes our work. 2 System Description In this section, we present the details of our En- glish emoji prediction system. In the dataset, each tweet corresponds to one label indicating one type of emojis. There are 20 types of emojis, most of which are emotions. We treat the problem as a multiclass predic- tion task and extract a variety of features. For the model GBM, besides the common features such as word n-gram features, we utilize extensive exter- nal resources to build diverse word clusters, lexi- con and syntactic features. For the model BLSTM, we adopt the pre-trained word embedding GloVe (Pennington et al., 2014). 2.1 Preprocessing As the first step, …

Content truncated to 500 words