Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention

Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention Ryosuke Miyazaki∗ Graduate School of System Design Tokyo Metropolitan University tmcit.miyazaki@gmail.com Mamoru Komachi Graduate School of System Design Tokyo Metropolitan University komachi@tmu.ac.jp Abstract Previous approaches to training syntax- based sentiment classification models required phrase-level annotated corpora, which are not readily available in many languages other than English. Thus, we propose the use of tree- structured Long Short-Term Memory with an attention mechanism that pays attention to each subtree of a parse tree. Experimental re- sults indicate that our model achieves state-of- the-art performance for a Japanese sentiment classification task. 1 Introduction Traditional approaches for sentiment classification rely on simple lexical features, such as a bag-of- words, that are ineffective for many sentiment clas- sification tasks (Pang et al., 2002). For example, the sentence “Insecticides kill pests.” contains both kill and pests, indicating negative polarity. But, the over- all expression is still deemed positive. To address this problem of polarity shift, Nak- agawa et al. (2010) presented a dependency-tree- based approach for the sentiment classification of a sentence. Their method assigns sentiment polarity to each subtree as a hidden variable that is not observ- able in the training data. The polarity of the overall sentence is then classified by a tree-conditional ran- dom field (Tree-CRF), marginalizing over the hid- den variables representing the polarities of the re- spective subtrees. In this manner, the model can handle polarity-shifting operations such as negation. However, this method suffers from feature sparse- ness because almost all features are combination features. ∗Now at Yahoo Japan Corporation. To overcome the data sparseness problem, deep- neural-network-based methods have attracted much attention because of their ability to use dense fea- ture representations (Socher et al., 2011; Socher et al., 2013; Kim, 2014; Kalchbrenner et al., 2014; Tai et al., 2015; Zhang and Komachi, 2015). In particu- lar, tree-structured approaches called recursive neu- ral networks (RvNNs) have been shown to perform well in sentiment classification tasks (Socher et al., 2011; Socher et al., 2013; Kim, 2014; Tai et al., 2015). Whereas Tree-CRF employs sparse and bi- nary feature representations, RvNNs avoid feature sparseness by learning dense and continuous fea- ture representations. However, annotation for each phrase is crucial for learning RvNN models, but there is no phrase-level annotated corpus in any lan- guage other than English. We therefore propose an RvNN model with an attention mechanism and augment the training ex- ample with polar dictionaries to compensate for the lack of phrase-level annotation. Although Kokkinos and Potamianos (2017) also provided an attention mechanism for phrase-level annotated corpus, our model performs well on a sentence-level annotated corpus through the introduction of polar dictionar- ies. The main contributions of this work are as fol- lows. • We show that RvNN models can be learned from a sentence-level polarity-tagged Japanese corpus using an attention mechanism and polar dictionaries. • We achieve the state-of-the-art performance in a Japanese sentiment classification task. PACLIC 32 466 32nd Pacific Asia Conference on Language, Information and Computation …

Content truncated to 500 words