Translingual Document Representations from Discriminative Projections

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 251–261, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Translingual Document Representations from Discriminative Projections John C. Platt Kristina Toutanova Microsoft Research 1 Microsoft Way Redmond, WA 98005, USA {jplatt,kristout,scottyih}@microsoft.com Wen-tau Yih Abstract Representing documents by vectors that are independent of language enhances machine translation and multilingual text categoriza- tion. We use discriminative training to create a projection of documents from multiple lan- guages into a single translingual vector space. We explore two variants to create these pro- jections: Oriented Principal Component Anal- ysis (OPCA) and Coupled Probabilistic Latent Semantic Analysis (CPLSA). Both of these variants start with a basic model of docu- ments (PCA and PLSA). Each model is then made discriminative by encouraging compa- rable document pairs to have similar vector representations. We evaluate these algorithms on two tasks: parallel document retrieval for Wikipedia and Europarl documents, and cross-lingual text classification on Reuters. The two discriminative variants, OPCA and CPLSA, significantly outperform their corre- sponding baselines. The largest differences in performance are observed on the task of re- trieval when the documents are only compa- rable and not parallel. The OPCA method is shown to perform best. 1 Introduction Given the growth of multiple languages on the In- ternet, Natural Language Processing must operate on dozens of languages. It is becoming critical that computers reach high performance on the following two tasks: • Comparable and parallel document re- trieval — Cross-language information retrieval and text categorization have become impor- tant with the growth of the Web (Oard and Diekema, 1998). In addition, machine trans- lation (MT) systems can be improved by training on sentences extracted from paral- lel or comparable documents mined from the Web (Munteanu and Marcu, 2005). Compa- rable documents can also be used for learning word-level translation lexicons (Fung and Yee, 1998; Rapp, 1999). • Cross-language text categorization — Appli- cations of text categorization, such as sentiment classification (Pang et al., 2002), are now re- quired to run on multiple languages. Catego- rization is usually trained on the language of the developer: it needs to be easily extended to other languages. There are two broad approaches to comparable document retrieval and cross-language text catego- rization. One approach is to translate queries or a training set from different languages into a single target language. Standard monolingual retrieval and classification algorithms can then be applied in the target language. Alternatively, a cross-language system can project a bag-of-words vector into a translingual lower- dimensional vector space. Ideally, vectors in this space represent the semantics of a document, inde- pendent of the language. The advantage of pre-translation is that MT sys- tems tend to preserve the meaning of documents. However, MT can be very slow (more than 1 second per document), preventing its use on large training sets. When full MT is not practical, a fast word-by- word translation model can be used instead, (Balles- teros and Croft, 1996) but may be less …

Content truncated to 500 words