Semi-Supervised Representation Learning for Cross-Lingual Text Classification

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1465–1475, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Semi-Supervised Representation Learning for Cross-Lingual Text Classification Min Xiao and Yuhong Guo Department of Computer and Information Sciences Temple University Philadelphia, PA 19122, USA {minxiao,yuhong}@temple.edu Abstract Cross-lingual adaptation aims to learn a pre- diction model in a label-scarce target lan- guage by exploiting labeled data from a label- rich source language. An effective cross- lingual adaptation system can substantially re- duce the manual annotation effort required in many natural language processing tasks. In this paper, we propose a new cross-lingual adaptation approach for document classifica- tion based on learning cross-lingual discrim- inative distributed representations of words. Specifically, we propose to maximize the log- likelihood of the documents from both lan- guage domains under a cross-lingual log- bilinear document model, while minimizing the prediction log-losses of labeled docu- ments. We conduct extensive experiments on cross-lingual sentiment classification tasks of Amazon product reviews. Our experimental results demonstrate the efficacy of the pro- posed cross-lingual adaptation approach. 1 Introduction With the rapid development of linguistic resources in different languages, developing cross-lingual nat- ural language processing (NLP) systems becomes increasingly important (Bel et al., 2003; Shanahan et al., 2004). Recently, cross-lingual adaptation methods have been studied to exploit labeled infor- mation from an existing source language domain where labeled training data is abundant for use in a target language domain where annotated training data is scarce (Prettenhofer and Stein, 2010). Pre- vious work has shown that cross-lingual adaptation can greatly reduce labeling effort for a variety of cross language NLP tasks such as document catego- rization (Bel et al., 2003; Amini et al., 2009), genre classification (Petrenz and Webber, 2012), and sen- timent classification (Shanahan et al., 2004; Wei and Pal, 2010; Prettenhofer and Stein, 2010). The fundamental challenge of cross-lingual adap- tation stems from a lack of overlap between the fea- ture space of the source language data and that of the target language data. To address this challenge, previous work in the literature mainly relies on au- tomatic machine translation tools. They first trans- late all the text data from one language domain into the other and then apply techniques such as domain adaptation (Wan et al., 2011; Rigutini and Maggini, 2005; Ling et al., 2008) and multi-view learning (Amini et al., 2009; Guo and Xiao, 2012b; Wan, 2009) to achieve cross-lingual adaptation. However, machine translation tools may not be freely available for all languages. Moreover, translating all the text data in one language into the other language is too time-consuming in reality. As an economic alter- native solution, cross-lingual representation learn- ing has recently been used in the literature to learn language-independent representations of the data for cross language text classification (Prettenhofer and Stein, 2010; Petrenz and Webber, 2012). In this paper, we propose to tackle cross language text classification by inducing cross-lingual predic- tive data representations with both labeled and un- labeled documents from the …

Content truncated to 500 words