MI&T Lab at SemEval-2017 task 4: An Integrated Training Method of Word Vector for Sentiment Classification

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 689–693, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics MI&T Lab at SemEval-2017 task 4: An Integrated Training Method of Word Vector for Sentiment Classification Jingjing Zhao, Yan Yang, Bing Xu Laboratory of Machine Intelligence and Translation, Harbin Institute of Technology School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China zhaojingjing@hit.edu.cn,yangyan@hit.edu.cn, hitxb@hit.edu.cn Abstract A CNN method for sentiment classifica- tion task in Task 4A of SemEval 2017 is presented. To solve the problem of word2vec training word vector slowly, a method of training word vector by inte- grating word2vec and Convolutional Neu- ral Network (CNN) is proposed. This training method not only improves the training speed of word2vec, but also makes the word vector more effective for the target task. Furthermore, the word2vec adopts a full connection between the input layer and the projection layer of the Con- tinuous Bag-of-Words (CBOW) for ac- quiring the semantic information of the original sentence. 1 Introduction The polarity of a Twitter message is classified into positive, negative and neutral in Twitter sentiment analysis. However, the difficulty of sentiment analysis greatly increases due to the ambiguity and the rhetorical of natural language (Liu, 2012). In recent years, the deep learning model has shown great potential in the task of sentiment classifica- tion (Socher et al., 2011; Poria et al., 2015; Socher et al., 2013). For short text data such as Twit- ter, Convolutional Neural Network (CNN) model (Kim, 2014; Dos Santos and Gatti, 2014; Chen et al., 2016) is the most widely and successfully used, and in the SemEval 2016-task4A competi- tion, the system ranked first also uses CNN model (Deriu et al., 2016). So CNN model is used to complete the task in our system. The task 4A of SemEval 20171 is a polarity classification task which requires participated systems to classify a 1http://alt.qcri.org/semeval2017/ task4/ given Twitter message into positive, negative or neutral (Rosenthal et al., 2017). The system integrates the word2vec and CNN to train the labeled data, generating the word vec- tor of each word in the data. This method can improve the training speed of word vector. In or- der to preserve the more semantic information of the original sentence effectively, the word2vec is fully connected between the input layer and the projection layer of the Continuous Bag-of-Words (CBOW). 2 System description 2.1 Word vector representation method Word2vec can represent every word that ap- pears in a large number of training texts as a lower dimension vector (usually 50-100 di- mensions). (Mikolov et al., 2013b; Rong, 2014; Mikolov et al., 2013a) have a detail descrip- tion of word2vec. Word2vec in our sys- tem uses Continuous Bag-of-Words (CBOW) model, and the structure is shown in Fig- ure 1, in which wi is a word, and the se- quence ⟨w(i−c), ..., w(i−1), w(i+1), ..., w(i+c)⟩ rep- resents the context of the word wi, and c is the window size. The word-vector length of the …

Content truncated to 500 words