Cross-Lingual Word Embeddings for Turkic Languages

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4054–4062 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4054 Cross-Lingual Word Embeddings for Turkic Languages Elmurod Kuriyozov∗, Yerai Doval†, Carlos G´omez-Rodr´ıguez∗ ∗Universidade da Coru˜na, CITIC Grupo LYS, Depto. de Computaci´on y Tecnolog´ıas de la Informaci´on Facultade de Inform´atica, Campus de Elvi˜na, A Coru˜na 15071, Spain {e.kuriyozov, carlos.gomez}@udc.es †Grupo COLE, Departamento de Inform´atica E.S. de Enxe˜nar´ıa Inform´atica, Universidade de Vigo Campus As Lagoas, Ourense 32004, Spain. yerai.doval@uvigo.es Abstract There has been an increasing interest in learning cross-lingual word embeddings to transfer knowledge obtained from a resource-rich language, such as English, to lower-resource languages for which annotated data is scarce, such as Turkish, Russian, and many others. In this paper, we present the first viability study of established techniques to align monolingual embedding spaces for Turkish, Uzbek, Azeri, Kazakh and Kyrgyz, members of the Turkic family which is heavily affected by the low-resource constraint. Those techniques are known to require little explicit supervision, mainly in the form of bilingual dictionaries, hence being easily adaptable to different domains, including low-resource ones. We obtain new bilingual dictionaries and new word embeddings for these languages and show the steps for obtaining cross-lingual word embeddings using state-of-the-art techniques. Then, we evaluate the results using the bilingual dictionary induction task. Our experiments confirm that the obtained bilingual dictionaries outperform previously-available ones, and that word embeddings from a low-resource language can benefit from resource-rich closely-related languages when they are aligned together. Furthermore, evaluation on an extrinsic task (Sentiment analysis on Uzbek) proves that monolingual word embeddings can, although slightly, benefit from cross-lingual alignments. Keywords: Less-Resourced/Endangered Languages, Multilinguality 1. Introduction Cross-lingual embeddings are continuous encodings of words or tokens from many languages into the same vector space. This implies, in principle, that words with similar semantics are represented by similar vectors irrespective of the language they come from. Previous work has shown that these continuous representations are useful to transfer knowledge from resource-rich languages, mainly English, to many other low-resource languages both in widespread use (Ruder et al., 2018; Doval et al., 2019b) and also threat- ened (Adams et al., 2017). An NLP task where knowledge transfer has been of particular interest is machine trans- lation (Zoph et al., 2016; Gu et al., 2018), but there is also work covering many other of the main tasks in the field: PoS tagging (Fang and Cohn, 2017), dependency parsing (Kulmizev, 2018), or sentiment analysis (Le et al., 2016). In this context, to our knowledge, no previous work has covered the case of Turkic languages, a family of languages estimated to be spoken by over 170 million speakers (Menz, 2016) with little to no NLP resources available. Even though the family comprises dozens of languages, of which at least five have over ten million speakers (Menz, 2016), only Turkish has a reasonable amount of publicly available resources that had been already used in low-resource sce- narios. For the rest of them, we had …

Content truncated to 500 words