INSIGHT-1 at SemEval-2016 Task 4: Convolutional Neural Networks for Sentiment Classification and Quantification

Proceedings of SemEval-2016, pages 178–182, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics INSIGHT-1 at SemEval-2016 Task 4: Convolutional Neural Networks for Sentiment Classification and Quantification Sebastian Ruder12 Parsa Ghaffari2 John G. Breslin1 1Insight Centre for Data Analytics National University of Ireland, Galway firstname.lastname@insight-centre.org 2Aylien Ltd. Dublin, Ireland firstname@aylien.com Abstract This paper describes our deep learning-based approach to sentiment analysis in Twitter as part of SemEval-2016 Task 4. We use a convolutional neural network to determine sentiment and participate in all subtasks, i.e. two-point, three-point, and five-point scale sentiment classification and two-point and five-point scale sentiment quantification. We achieve competitive results for two-point scale sentiment classification and quantifica- tion, ranking fifth and a close fourth (third and second by alternative metrics) respec- tively despite using only pre-trained embed- dings that contain no sentiment information. We achieve good performance on three-point scale sentiment classification, ranking eighth out of 35, while performing poorly on five- point scale sentiment classification and quan- tification. An error analysis reveals that this is due to low expressiveness of the model to capture negative sentiment as well as an in- ability to take into account ordinal informa- tion. We propose improvements in order to address these and other issues. 1 Introduction Social media allows hundreds of millions of people to interact and engage with each other, while ex- pressing their thoughts about the things that move them. Sentiment analysis (Pang and Lee, 2008) allows us to gain insights about opinions towards persons, objects, and events in the public eye and is used nowadays to gauge public opinion towards companies or products, to analyze customer satis- faction, and to detect trends. Its immediacy allowed Twitter to become an im- portant platform for expressing opinions and public discourse, while the accessibility of large quantities of data in turn made it the focal point of social media sentiment analysis research. Recently, deep learning-based approaches have demonstrated remarkable results for text classifi- cation and sentiment analysis (Kim, 2014) and have performed well for phrase-level and message- level sentiment classification (Severyn and Mos- chitti, 2015). Past SemEval competitions in Twitter sentiment analysis (Rosenthal et al., 2014; Rosenthal et al., 2015) have contributed to shape research in this field. SemEval-2016 Task 4 (Nakov et al., 2016) is no exception, as it introduces both quantification and five-point-scale classification tasks, neither of which have been tackled with deep learning-based approaches before. We apply our deep learning-based model for sen- timent analysis to all subtasks of SemEval-2016 Task 4: three-point scale message polarity classi- fication (subtask A), two-point and five-point scale topic sentiment classification (subtasks B and C re- spectively), and two-point and five-point scale topic sentiment quantification (subtasks D and E respec- tively). Our model achieves excellent results for subtasks B and D, ranks competitively for subtask A, while performing poorly for subtasks C and E. We perform an error analysis of our model to obtain a better un- derstanding of strengths and weaknesses of a deep learning-based approach particularly for these …

Content truncated to 500 words