Inverted indexing for cross-lingual NLP

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1713–1722, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Inverted indexing for cross-lingual NLP Anders Søgaard∗ ˇZeljko Agi´c∗ H´ector Mart´ınez Alonso∗ Barbara Plank∗ Bernd Bohnet† Anders Johannsen∗ ∗Center for Language Technology, University of Copenhagen, Denmark †Google, London, United Kingdom soegaard@hum.ku.dk Abstract We present a novel, count-based approach to obtaining inter-lingual word represen- tations based on inverted indexing of Wikipedia. We present experiments ap- plying these representations to 17 datasets in document classification, POS tagging, dependency parsing, and word alignment. Our approach has the advantage that it is simple, computationally efficient and almost parameter-free, and, more im- portantly, it enables multi-source cross- lingual learning. In 14/17 cases, we im- prove over using state-of-the-art bilingual embeddings. 1 Introduction Linguistic resources are hard to come by and un- evenly distributed across the world’s languages. Consequently, transferring linguistic resources or knowledge from one language to another has been identified as an important research problem. Most work on cross-lingual transfer has used English as the source language. There are two reasons for this; namely, the availability of English re- sources and the availability of parallel data for (and translations between) English and most other languages. In cross-lingual syntactic parsing, for exam- ple, two approaches to cross-lingual learning have been explored, namely annotation projec- tion and delexicalized transfer. Annotation pro- jection (Hwa et al., 2005) uses word-alignments in human translations to project predicted source- side analyses to the target language, producing a noisy syntactically annotated resource for the tar- get language. On the other hand, delexicalized transfer (Zeman and Resnik, 2008; McDonald et al., 2011; Søgaard, 2011) simply removes lexi- cal features from mono-lingual parsing models, but assumes reliable POS tagging for the target language. Delexicalized transfer works particu- larly well when resources from several source lan- guages are used for training; learning from mul- tiple other languages prevents over-fitting to the peculiarities of the source language. Some au- thors have also combined annotation projection and delexicalized transfer, e.g., McDonald et al. (2011). Others have tried to augment delexical- ized transfer models with bilingual word repre- sentations (T¨ackstr¨om et al., 2013; Xiao and Guo, 2014). In cross-lingual POS tagging, mostly annotation projection has been explored (Fossum and Abney, 2005; Das and Petrov, 2011), since all features in POS tagging models are typically lexical. How- ever, using bilingual word representations was re- cently explored as an alternative to projection- based approaches (Gouws and Søgaard, 2015). The major drawback of using bi-lexical repre- sentations is that it limits us to using a single source language. T¨ackstr¨om et al. (2013) ob- tained significant improvements using bilingual word clusters over a single source delexicalized transfer model, for example, but even better re- sults were obtained with delexicalized transfer in McDonald et al. (2011) by simply using several source languages. This paper introduces a simple method for ob- taining truly inter-lingual word representations in order to …

Content truncated to 500 words