UNIMELB at SemEval-2016 Tasks 4A and 4B: An Ensemble of Neural Networks and a Word2Vec Based Model for Sentiment Classification

Proceedings of SemEval-2016, pages 183–189, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics UNIMELB at SemEval-2016 Tasks 4A and 4B: An Ensemble of Neural Networks and a Word2Vec Based Model for Sentiment Classification XingYi Xu HuiZhi Liang Timothy Baldwin Department of Computing and Information Systems The University of Melbourne VIC 3010, Australia stevenxxiu@gmail.com oklianghuizi@gmail.com tb@ldwin.net Abstract This paper describes our sentiment clas- sification system for microblog-sized doc- uments, and documents where a topic is present. The system consists of a soft- voting ensemble of a word2vec language model adapted to classification, a convolu- tional neural network (CNN), and a long- short term memory network (LSTM). Our main contribution consists of a way to in- troduce topic information into this model, by concatenating a topic embedding, con- sisting of the averaged word embedding for that topic, to each word embedding vector in our neural networks. When we apply our models to SemEval 2016 Task 4 sub- tasks A and B, we demonstrate that the ensemble performed better than any sin- gle classifier, and our method of includ- ing topic information achieves a substan- tial performance gain. According to re- sults on the official test sets, our model ranked 3rd for 𝐴��� PNin the message-only subtask A (among 34 teams) and 1st for accuracy on the topic-dependent subtask B (among 19 teams). 1 Introduction The rapid growth of user-generated content, much of which is sentiment-laden, has fueled an interest in sentiment analysis (Pang and Lee, 2008; Liu, 2010). One popular form of senti- ment analysis involves classifying a document into discrete classes, depending on whether it expresses positive or negative sentiment (or nei- ther). The classification can also be dependent upon a particular topic. In this work, we de- scribe the method we used for the sentiment classification of tweets, with or without a topic. Our approach to the document classification task consists of an ensemble of 3 classifiers via soft-voting, 2 of which are neural network mod- els. One is the convolutional neural network (CNN) architecture of Kim (2014), and another is a Long Short Term Memory (LSTM)-based network (Hochreiter and Schmidhuber, 1997). Both were first tuned on a distant-labelled data set. The third classifier adapted word2vec to output classification probabilities using Bayes’ formula, a slightly modified version of Taddy (2015). Despite the word2vec classifier being in- tended as a baseline, and having a small weight in the ensemble, it proved crucial for the ensem- ble to work well. To adapt our models to the case where a topic is present, in the neural net- work models, we concatenated the embedding vectors for each word with a topic embedding, which consisted of the element-wise average of all word vectors in a particular topic. We applied our approach to SemEval 2016 Task 4, including the message-only subtask (Task A) and the topic-dependent subtask (Task B) (Nakov et al., to appear). Our model ranked third for 𝐴��� PNin the message-only subtask A (among 34 teams) and first for accuracy1 on …

Content truncated to 500 words