UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 464–469, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification Aliaksei Severyn DISI, University of Trento 38123 Povo (TN), Italy severyn@disi.unitn.it Alessandro Moschitti Qatar Computing Research Institue 5825 Doha, Qatar amoschitti@qf.org.qa Abstract This paper describes our deep learning system for sentiment analysis of tweets. The main contribution of this work is a process to ini- tialize the parameter weights of the convolu- tional neural network, which is crucial to train an accurate model while avoiding the need to inject any additional features. Briefly, we use an unsupervised neural language model to ini- tialize word embeddings that are further tuned by our deep learning model on a distant super- vised corpus. At a final stage, the pre-trained parameters of the network are used to initialize the model which is then trained on the super- vised training data from Semeval-2015. Ac- cording to results on the official test sets, our model ranks 1st in the phrase-level subtask A (among 11 teams) and 2nd on the message- level subtask B (among 40 teams). Interest- ingly, computing an average rank over all six test sets (official and five progress test sets) puts our system 1st in both subtasks A and B. 1 Introduction In this work we describe our deep convolutional neu- ral network for sentiment analysis of tweets. Its architecture is most similar to the deep learning systems presented in (Kalchbrenner et al., 2014; Kim, 2014) that have recently established new state- of-the-art results on various NLP sentence clas- sification tasks also including sentiment analysis. While already demonstrating excellent results, train- ing a convolutional neural network that would beat hand-engineered approaches that also rely on multi- ple manual and automatically constructed lexicons, e.g. (Mohammad et al., 2013; Xiaodan Zhu, 2014; Severyn and Moschitti, 2015), requires careful at- tention. This becomes an even harder problem espe- cially in cases when the amount of labelled data is relatively small, e.g., thousands of examples. Turns out, providing the network with good ini- tialisation parameters makes all the difference in training an accurate model. We propose a three-step process we follow to train our deep learning model for sentiment classification. It can be summarized as follows: (i) word embeddings are initialized using a neural language model (Ronan Collobert, 2008; Mikolov et al., 2013) which is trained on a large un- supervised collection of tweets; (ii) we use our con- volutional neural network to further refine the em- beddings on a large distant supervised corpus (Go et al., 2009); (iii) the word embeddings and other parameters of the network obtained at the previous stage are used to initialize the network that is then trained on a supervised corpus from Semeval-2015. We apply our deep learning model on two sub- tasks of Semeval-2015 Twitter Sentiment Analysis (Task 10) challenge (Rosenthal et al., 2015): phrase- level (subtask A) and message-level (subtask B). Our system ranks 1st on …

Content truncated to 500 words