Neural Transduction for Multilingual Lexical Translation

Proceedings of the 28th International Conference on Computational Linguistics, pages 4373–4384 Barcelona, Spain (Online), December 8-13, 2020 4373 Neural Transduction for Multilingual Lexical Translation Dylan Lewis and Winston Wu and Arya D. McCarthy and David Yarowsky Department of Computer Science Johns Hopkins University {dlewis77,wswu,arya,yarowsky}@jhu.edu Abstract We present a method for completing multilingual translation dictionaries. Our probabilistic ap- proach can synthesize new word forms, allowing it to operate in settings where correct translations have not been observed in text (cf. cross-lingual embeddings). In addition, we propose an approxi- mate Maximum Mutual Information (MMI) decoding objective to further improve performance in both many-to-one and one-to-one word level translation tasks where we use either multiple input languages for a single target language or more typical single language pair translation. The model is trained in a many-to-many setting, where it can leverage information from related languages to predict words in each of its many target languages. We focus on 6 languages: French, Spanish, Italian, Portuguese, Romanian, and Turkish. When indirect multilingual information is available, ensembling with mixture-of-experts as well as incorporating related languages leads to a 27% relative improvement in whole-word accuracy of predictions over a single-source baseline. To seed the completion when multilingual data is unavailable, it is better to decode with an MMI objective. 1 Introduction Translation matrices, i.e. concept-aligned word lists across the world’s languages (Buck, 1949; Swadesh, 1950; Swadesh, 1952; Swadesh, 1955; Swadesh, 1971; Nastase and Strube, 2013; Wu et al., 2018), enable several avenues of exploration in computational linguistics and human language technologies. They strengthen word alignment models, which can in turn be useful for machine translation (Garg et al., 2019), robust projection of morphosyntactic information across alignments (Yarowsky and Ngai, 2001), and interlinear glossing. Further, fuller word lists enable neogrammarians to better explore phylogeny and phonology across languages (Hewson, 1973; Lowe and Mazaudon, 1994). This work is motivated by the tremendous capacity for humans to generalize during translation, producing forms for words that have not been seen before. This becomes valuable especially for lower- frequency words, which may not have been observed in training data but could be inferrable through regular processes such as cognate relationships with related languages (Mulloni, 2007; Beinborn et al., 2013), borrowing from neighboring or other influential languages, and even esoteric features like temporal similarity (Schafer and Yarowsky, 2002; Wijaya et al., 2017) or image similarity (Bergsma and Van Durme, 2011). In this work, we focus on these cognate relationships, because cognates form a large amount of both core vocabulary (Wu et al., 2020) and technical language (Mulloni, 2007).1 Unlike conventional bilingual lexicon induction (Rapp, 1995), we do not wish to limit the predictions to words that have been previously seen in a corpus. Automated methods to induce plausible translations for lexical translation would significantly reduce the human effort needed for both elicitation (Chelliah, 2001) and building machine translation systems for less heavily supported languages. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 1We use a …

Content truncated to 500 words