An Ensemble Method with Sentiment Features and Clustering Support

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 644–653, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP An Ensemble Method with Sentiment Features and Clustering Support Nguyen Huy Tien Japan Advanced Institute of Science and Technology (JAIST) ntienhuy@jaist.ac.jp Nguyen Minh Le Japan Advanced Institute of Science and Technology (JAIST) nguyenml@jaist.ac.jp Abstract Deep learning models have recently been applied successfully in natural language processing, especially sentiment analysis. Each deep learning model has a particu- lar advantage, but it is difficult to com- bine these advantages into one model, es- pecially in the area of sentiment analy- sis. In our approach, Convolutional Neu- ral Network (CNN) and Long Short Term Memory (LSTM) were utilized to learn sentiment-specific features in a freezing scheme. This scenario provides a novel and efficient way for integrating advan- tages of deep learning models. In addi- tion, we also grouped documents into clus- ters by their similarity and applied the pre- diction score of Naive Bayes SVM (NB- SVM) method to boost the classification accuracy of each group. The experiments show that our method achieves the state- of-the-art performance on two well-known datasets: IMDB large movie reviews for document level and Pang & Lee movie re- views for sentence level. 1 Introduction The emergence of web 2.0, which allows users to generate content, is causing the rapid increase in the amount of data. This platform, which enables millions of users to share information and com- ments, has a high demand for extracting knowl- edge from user-generated content. An important information to be analyzed from those comments is opinions/sentiments, which express subjective opinions of particular users. Sentiment analysis is a fundamental task and has attracted a huge amount of research in recent years (Pang and Lee, 2008; Liu, 2012). The task calls for identifying the sentiment polarity (positive, negative) of a com- ment or review. Wang (2012) used a Support Vector Machine variant with Naive Bayes feature (NBSVM). Pre- senting a document or a sentence with Bag of bi-gram features, NBSVM consistently performs well across datasets of long and short reviews. Recently, the success of deep learning in natu- ral language processing has led to many efficient methods for sentiment analysis such as Paragraph Vector (Le and Mikolov, 2014), CNN (Kalch- brenner et al., 2014; Kim, 2014; Zhang and Wal- lace, 2015), LSTM (Wang et al., 2015; Liu et al., 2015). In Paragraph Vector, Le and Mikolov em- ployed the technique of Word embedding repre- sentation using neural networks (Bengio et al., 2003; Collobert and Weston, 2008; Mnih and Hin- ton, 2009; Turian et al., 2010; Mikolov et al., 2013) to represent a document or paragraph as a vector. This document modeling outperformed the Bag of Words model in sentiment analysis and in- formation retrieval. Li (2015) has enhanced the architecture of Paragraph Vector by allowing the model to predict not only words but also n-gram features (DVngram). CNN is capable of capturing local relationships between neighbor words in a sentence but …

Content truncated to 500 words