Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification

Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification Xilun Chen† xlchen@cs.cornell.edu Yu Sun† ys646@cornell.edu Ben Athiwaratkun‡ pa338@cornell.edu Claire Cardie† cardie@cs.cornell.edu Kilian Weinberger† kqw4@cornell.edu †Department of Computer Science, Cornell University ‡Department of Statistical Science, Cornell University Abstract In recent years great success has been achieved in sentiment classification for En- glish, thanks in part to the availability of copious annotated resources. Unfortunately, most languages do not enjoy such an abun- dance of labeled data. To tackle the senti- ment classification problem in low-resource languages without adequate annotated data, we propose an Adversarial Deep Averaging Network (ADAN1) to transfer the knowledge learned from labeled data on a resource-rich source language to low-resource languages where only unlabeled data exist. ADAN has two discriminative branches: a sentiment classifier and an adversarial language dis- criminator. Both branches take input from a shared feature extractor to learn hidden rep- resentations that are simultaneously indica- tive for the classification task and invariant across languages. Experiments on Chinese and Arabic sentiment classification demon- strate that ADAN significantly outperforms state-of-the-art systems. 1 Introduction Many state-of-the-art models for sentiment classi- fication (Socher et al., 2013; Iyyer et al., 2015; Tai et al., 2015) are supervised learning approaches that rely on the availability of an adequate amount of labeled training data. For a few resource-rich languages, including English, such labeled data are indeed available. For the vast majority of lan- guages, however, it is the norm that only a lim- ited amount of annotated text exists. Worse still, many low-resource languages have no labeled data at all. 1The source code of ADAN is available at https:// github.com/ccsasuke/adan. To aid the creation of sentiment classification systems in such low-resource languages, an active research direction is cross-lingual sentiment clas- sification (CLSC), in which the abundant re- sources of a source language (likely English, denoted as SOURCE) are leveraged to produce sen- timent classifiers for a target language (TARGET). In general, CLSC methods make use of general- purpose bilingual resources—such as hand-crafted bilingual lexica or parallel corpora—to alleviate or eliminate the need for task-specific TARGET an- notations. In particular, the bilingual resource of choice for the majority of previous CLSC models is a full-fledged Machine Translation (MT) sys- tem (Wan, 2008, 2009; Lu et al., 2011; Zhou et al., 2016), a component that is expensive to obtain. In this work, we propose a language-adversarial training approach that does not need a highly engineered MT system, and requires orders of magnitude less in terms of the size of a parallel corpus. Specifically, we propose an Adversarial Deep Averaging Network (ADAN) that leverages a set of bilingual word embeddings (BWEs; Zou et al., 2013) trained on bitexts, in order to elimi- nate the need for labeled TARGET training data.2 We introduce the ADAN model in §2, and in §3 evaluate ADAN using English as the SOURCE with two TARGET choices: Chinese and Arabic. ADAN is first compared to two baseline systems: (i) one trained only on labeled SOURCE data, relying on BWEs for cross-lingual generalization; and …

Content truncated to 500 words