Language adaptation experiments via cross-lingual embeddings for related languages

Language adaptation experiments via cross-lingual embeddings for related languages Serge Sharoff Centre for Translation Studies, University of Leeds, LS2 9JT, UK s.sharoff@leeds.ac.uk Abstract Language Adaptation (similarly to Domain Adaptation) is a general approach to extend existing resources from a better resourced language (donor) to a lesser resourced one (recipient) by exploiting the lexical and grammatical similarity between them when the two languages are related. The current study improves the state of the art in cross-lingual word embeddings by considering the impact of orthographic similarity between cognates. In particular, the use of the Weighted Levenshtein Distance combined with orthogonalisation of the translation matrix and generalised correction for hubness can considerably improve the state of the art in induction of bilingual lexicons. In addition to intrinsic evaluation in the bilingual lexicon induction task, the paper reports extrinsic evaluation of the cross-lingual embeddings via their application to the Named-Entity Recognition task across Slavonic languages. The tools and the aligned word embedding spaces for the Romance and Slavonic language families have been released. Keywords: Word embeddings, Related languages, Cognate words, Comparable corpora 1. Introduction Parallel corpora play an important role in many multi- lingual NLP applications, such as Machine Translation, Cross-Lingual Text Classification or Information Retrieval. However, the topics and genres of parallel corpora are lim- ited even for better resourced languages, e.g., resources are scarcer outside of the official documents of Europarl and the United Nations (Koehn, 2005; Eisele and Chen, 2010). Also, even if each individual language has reasonably good parallel resources, such as Polish and Russian aligned with English, it is difficult to find a large parallel corpus, which contains this specific, e.g., Polish-Russian, language pair. Monolingual corpora can be substantially bigger and more varied in comparison to parallel ones. Comparable corpora of different levels of comparability (Sharoff et al., 2013) can be used for induction of bilingual lexicons from small seed dictionaries. The present paper follows an influential study (Mikolov et al., 2013), which presented a method for building multilingual embedding spaces. In addition to a model with a seed bilingual dictionary, it also introduced constraints on what its authors call “morphological struc- ture” (actually the Levenshtein Distance) for keeping only the cognate words in the output. However, further work on bilingual lexicon induction did not include the use of cognates, especially in the context of related languages. The importance of utilising links be- tween related languages can be illustrated by the use of Machine Translation via a pivot language. A simple dic- tionary transfer from Ukrainian into Russian followed by MT for the better resourced Russian-English pair easily beats MT translating from Ukrainian directly into English using far smaller resources (Babych et al., 2007). Over- all, many lesser resourced languages can benefit from Lan- guage Adaptation by applying the models developed for the better resourced ones. The present study advances the state of the art by com- bining existing techniques of building cross-lingual em- bedding from comparable corpora with the Weighted Lev- enshtein distance, when the weights are themselves …

Content truncated to 500 words