Learning Bilingual Sentiment-Specific Word Embeddings without Cross-lingual Supervision

Proceedings of NAACL-HLT 2019, pages 420–429 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 420 Learning Bilingual Sentiment-Specific Word Embeddings without Cross-lingual Supervision Yanlin Feng and Xiaojun Wan Institute of Computer Science and Technology, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {fengyanlin,wanxiaojun}@pku.edu.cn Abstract Word embeddings learned in two languages can be mapped to a common space to pro- duce Bilingual Word Embeddings (BWE). Un- supervised BWE methods learn such a map- ping without any parallel data. However, these methods are mainly evaluated on tasks of word translation or word similarity. We show that these methods fail to capture the sentiment information and do not perform well enough on cross-lingual sentiment anal- ysis. In this work, we propose UBiSE (Un- supervised Bilingual Sentiment Embeddings), which learns sentiment-specific word repre- sentations for two languages in a common space without any cross-lingual supervision. Our method only requires a sentiment corpus in the source language and pretrained mono- lingual embeddings of both languages. We evaluate our method on three language pairs for cross-lingual sentiment analysis. Exper- imental results show that our method out- performs previous unsupervised BWE meth- ods and even supervised BWE methods. Our method succeeds for a distant language pair English-Basque. 1 Introduction Lack of annotated corpora degrades the qual- ity of sentiment analysis in low-resource lan- guages. Cross-lingual sentiment analysis tackles this problem by adapting the sentiment resource in a resource-rich language (the source language) to a resource-poor language (the target language). Bilingual Word Embeddings (BWE) provide a way to transfer the sentiment information from the source language to the target language. There has been an increasing interest in BWE methods in re- cent years, including both supervised methods and unsupervised methods. Supervised BWE meth- ods map the word vectors of the two languages in a common space by exploiting either a bilingual seed dictionary or other parallel data, while unsu- pervised BWE methods do not utilize any form of bilingual supervision. Yet, these methods are mostly evaluated on tasks of word translation or word similarity, and do not perform well enough on cross-lingual sentiment analysis as shown in Section 4. Consider the case where we want to perform sentiment analysis on the target language with merely an annotated sentiment corpus in the source language. We assume pretrained monolin- gual embeddings of both languages are available to us. One solution is to first align the embeddings of both languages in a common space using un- supervised BWE methods, then train a classifier based on the source sentiment corpus. In this so- lution, no sentiment information is utilized to learn the alignment. In this paper, we propose to exploit the sen- timent information and learn sentiment-specific alignment. The sentiment information is gradu- ally incorporated into the BWE through an itera- tive constraint relaxation procedure. Unlike pre- vious work which performed alignment in a single direction by linearly mapping the source vectors to the target vector space, we propose an alignment model that maps …

Content truncated to 500 words