Japanese Sentiment Classification with Stacked Denoising Auto-Encoder using Distributed Word Representation

Japanese Sentiment Classification with Stacked Denoising Auto-Encoder using Distributed Word Representation Peinan Zhang Graduate School of System Design Tokyo Metropolitan University zhang-peinan@ed.tmu.ac.jp Mamoru Komachi Graduate School of System Design Tokyo Metropolitan University komachi@tmu.ac.jp Abstract Traditional sentiment classification methods often require polarity dictionaries or crafted features to utilize machine learning. How- ever, those approaches incur high costs in the making of dictionaries and/or features, which hinder generalization of tasks. Ex- amples of these approaches include an ap- proach that uses a polarity dictionary that can- not handle unknown or newly invented words and another approach that uses a complex model with 13 types of feature templates. We propose a novel high performance sentiment classification method with stacked denoising auto-encoders that uses distributed word rep- resentation instead of building dictionaries or utilizing engineering features. The results of experiments conducted indicate that our model achieves state-of-the-art performance in Japanese sentiment classification tasks. 1 Introduction As the popularity of social media continues to rise, serious attention is being given to review informa- tion nowadays. Reviews with positive/negative rat- ings, in particular, help (potential) customers with product comparisons and to make purchasing deci- sions. Consequently, automatic classification of the polarities (such as positive and negative) of reviews is extremely important. Traditional approaches to sentiment analysis uti- lize polarity dictionaries or classification rules. Al- though these approaches are fairly accurate, they depend on languages that may require significant amounts of manual labor. Further, dictionary-based methods have difficulty dealing with new or un- known words. Machine learning-based methods are widely adopted in sentiment classification in order to miti- gate the problems associated with the making of dic- tionaries and/or rules. One of the most basic features used in machine learning-based sentiment classifi- cation is the bag-of-words feature (Wang and Man- ning, 2012; Pang et al., 2002). In machine learning- based frameworks, the weights of words are auto- matically learned from a training corpus instead of being manually assigned. However, the bag-of-words feature cannot take syntactic structures into account. This leads to mis- takes such as “a great design but inconvenient” and “inconvenient but a great design” being deemed to have the same meaning, even though their nu- ances are different; the former is somewhat nega- tive whereas the latter is slightly positive. To solve this syntactic problem, Nakagawa et al. (2010) pro- posed a sentiment analysis model that used depen- dency trees with polarities assigned to their subtrees. However, their proposed model requires specialized knowledge to design complicated feature templates. In this study, we propose an approach that uses distributed word representation to overcome the first problem and deep neural networks to alleviate the second problem. The former is an unsupervised method capable of representing a word ’s meaning without using hand-tagged resources such as a po- larity dictionary. In addition, it is robust to the data sparseness problem. The latter is a highly expressive model that does not utilize complex engineering fea- tures or models. Our research makes the following two main con- tributions: PACLIC 29 …

Content truncated to 500 words