Attention-based LSTM Network for Cross-Lingual Sentiment Classification

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 247–256, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Attention-based LSTM Network for Cross-Lingual Sentiment Classification Xinjie Zhou, Xiaojun Wan and Jianguo Xiao Institute of Computer Science and Technology, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {zhouxinjie,wanxiaojun,xiaojianguo}@pku.edu.cn Abstract Most of the state-of-the-art sentiment classifi- cation methods are based on supervised learn- ing algorithms which require large amounts of manually labeled data. However, the labeled resources are usually imbalanced in different languages. Cross-lingual sentiment classification tackles the problem by adapting the sentiment resources in a resource-rich language to resource-poor languages. In this study, we propose an attention-based bilingual representation learning model which learns the distributed semantics of the documents in both the source and the target languages. In each language, we use Long Short Term Mem- ory (LSTM) network to model the documents, which has been proved to be very effective for word sequences. Meanwhile, we propose a hierarchical attention mechanism for the bilingual LSTM network. The sentence-level attention model learns which sentences of a document are more important for determining the overall sentiment while the word-level attention model learns which words in each sentence are decisive. The proposed model achieves good results on a benchmark dataset using English as the source language and Chinese as the target language. 1 Introduction Most of the sentiment analysis research focuses on sentiment classification which aims to determine whether the users attitude is positive, neutral or negative. There are two classes of mainstreaming sentiment classification algorithms: unsupervised methods which usually require a sentiment lexicon (Taboada et al., 2011) and supervised methods (Pang et al., 2002) which require manually labeled data. However, both of these sentiment resources are unbalanced in different languages. The sentiment lexicon or labeled data are rich in several languages such as English and are poor in others. Manually building these resources for all the languages will be expensive and time-consuming. Cross-lingual sentiment classification tackles the problem by try- ing to adapt the resources in one language to other languages. It can also be regarded as a special kind of cross-lingual text classification task. Recently, there have been several bilingual rep- resentation learning methods such as (Hermann and Blunsom, 2014; Gouws et al., 2014) for cross-lingual sentiment or text classification which achieve promising results. They try to learn a joint embedding space for different languages such that the training data in the source language can be directly applied to the test data in the target language. However, most of the studies only use simple functions, e.g. arithmetic average, to synthesize representations for larger text sequences. Some of them use more complicated compositional models such as the bi-gram non-linearity model in (Hermann and Blunsom, 2014) which also fail to capture the long distance dependencies in texts. In this study, we propose an attention-based bilingual LSTM network for cross-lingual sentiment classification. LSTMs have been proved to be very effective to model word sequences and are …

Content truncated to 500 words