BB_twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 573–580, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics BB twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs Mathieu Cliche Bloomberg mcliche@bloomberg.net Abstract In this paper we describe our attempt at producing a state-of-the-art Twitter senti- ment classifier using Convolutional Neural Networks (CNNs) and Long Short Term Memory (LSTMs) networks. Our sys- tem leverages a large amount of unlabeled data to pre-train word embeddings. We then use a subset of the unlabeled data to fine tune the embeddings using distant su- pervision. The final CNNs and LSTMs are trained on the SemEval-2017 Twitter dataset where the embeddings are fined tuned again. To boost performances we ensemble several CNNs and LSTMs to- gether. Our approach achieved first rank on all of the five English subtasks amongst 40 teams. 1 Introduction Determining the sentiment polarity of tweets has become a landmark homework exercise in natural language processing (NLP) and data science classes. This is perhaps because the task is easy to understand and it is also easy to get good results with very simple methods (e.g. positive - negative words counting). The practical applications of this task are wide, from monitoring popular events (e.g. Presidential debates, Oscars, etc.) to extracting trading signals by monitoring tweets about public companies. These applications often benefit greatly from the best possible accuracy, which is why the SemEval-2017 Twitter competition promotes research in this area. The competition is divided into five subtasks which involve standard classification, ordinal classifi- cation and distributional estimation. For a more detailed description see (Rosenthal et al., 2017). In the last few years, deep learning techniques have significantly out-performed traditional meth- ods in several NLP tasks (Chen and Manning, 2014; Bahdanau et al., 2014), and sentiment anal- ysis is no exception to this trend (Rojas-Barahona, 2016). In fact, previous iterations of the Se- mEval Twitter sentiment analysis competition have already established their power over other approaches (Nakov et al., 2016; Severyn and Mos- chitti, 2015; Deriu et al., 2016). Two of the most popular deep learning techniques for sentiment analysis are CNNs and LSTMs. Consequently, in an effort to build a state-of-the-art Twitter sentiment classifier, we explore both models and build a system which combines both. This paper is organized as follows. In sec. 2 we describe the architecture of the CNN and the LSTM used in our system. In sec. 3 we expand on the three training phases used in our system. In sec. 4 we discuss the various tricks that were used to fine tune the system for each individual subtasks. Finally in sec. 5 we present the perfor- mance of the system and in sec. 6 we outline our main conclusions. 2 System description 2.1 CNN Let us now describe the architecture of the CNN we worked with. Its architecture is almost identi- cal to the CNN of Kim (2014). A smaller version of our model is illustrated on Fig. 1. …

Content truncated to 500 words