pkudblab at SemEval-2016 Task 6 : A Specific Convolutional Neural Network System for Effective Stance Detection

Proceedings of SemEval-2016, pages 384–388, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics pkudblab at SemEval-2016 Task 6 : A Specific Convolutional Neural Network System for Effective Stance Detection Wan Wei, Xiao Zhang, Xuqin Liu, Wei Chen, Tengjiao Wang Key Laboratory of High Confidence Software Technologies, Ministry of Education, School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China {wanw, zxcs, xuqinliu, pekingchenwei, tjwang}@pku.edu.cn Abstract In this paper, we develop a convolutional neu- ral network for stance detection in tweets. Ac- cording to the official results, our system ranks 1st on subtask B (among 9 teams) and ranks 2nd on subtask A (among 19 teams) on the twitter test set of SemEval2016 Task 6. The main contribution of our work is as follows. We design a ”vote scheme” for prediction in- stead of predicting when the accuracy of vali- dation set reaches its maximum. Besides, we make some improvement on the specific sub- tasks. For subtask A, we separate datasets into five sub-datasets according to their tar- gets, and train and test five separate models. For subtask B, we establish a two-class train- ing dataset from the official domain corpus, and then modify the softmax layer to perform three-class classification. Our system can be easily re-implemented and optimized for other related tasks. 1 Introduction There are several requirements for stance detect- ing applications on the internet. However it is un- practical for humans to classify massive amounts of tweets. Twitter stance detection aims to automat- ically determine the emotional tendency of tweets. To classify tweets polarity, mainstream approaches are based on Pang (Pang et al., 2002), like regression problem, using machine learning algorithm to build classifiers from tweets with manually annotated po- larity to classify the polarity of a tweet(Jiang et al., 2011; Hu et al., 2013; Dong et al., 2014). In this direction, most studies focus on designing effective features to obtain better classification performance (Pang and Lee, 2008; Liu, 2012; Murakami and Raymond, 2010). For example, Mohammad (Mo- hammad and Turney, 2013) implements some sen- timent lexicons and several manually-selected fea- tures. To leverage massive tweets containing pos- itive and negative emoticons for automatically fea- ture learning, Tang (Tang et al., 2014) proposes to learn sentiment-specific word embedding. We trans- fer this method to detect tweets stance. In this paper, we develop a specific convolu- tional neural network learning model for stance de- tection. Firstly, we learn word embedding from Google News database as the input of our system. Afterwards, we train the CNN model with the Se- mEval2016 Task 6 dataset. Finally, we design a ”vote scheme” using the softmax results to predict the label of test set. We also make some task specific improvement. For subtask A, we separate datasets into five sub-dataset, and train and test five sepa- rate models. For subtask B, we establish a two- class training dataset from the official domain cor- pus based on several special expressions. We eval- uate our deep learning system on the …

Content truncated to 500 words