Pivot Translation for Zero-resource Language Pairs Based on a Multilingual Pretrained Model

Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 348–359 September 4–8, 2023, Macau SAR, China. ©2023 The authors. This article is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). 348 Pivot Translation for Zero-resource Language Pairs Based on a Multilingual Pretrained Model Kenji Imamura kenji.imamura@nict.go.jp Masao Utiyama mutiyama@nict.go.jp Eiichiro Sumita eiichiro.sumita@nict.go.jp National Institute of Information and Communications Technology, Hikari-dai, Seika- cho, Soraku-gun, Kyoto, 619-0289, Japan Abstract A multilingual translation model enables a single model to handle multiple languages. How- ever, the translation qualities of unlearned language pairs (i.e., zero-shot translation qualities) are still poor. By contrast, pivot translation translates source texts into target ones via a pivot language such as English, thus enabling machine translation without parallel texts between the source and target languages. In this paper, we perform pivot translation using a multilingual model and compare it with direct translation. We improve the translation quality without using parallel texts of direct translation by fine-tuning the model with machine-translated pseudo-translations. We also discuss what type of parallel texts are suitable for effectively improving the translation quality in multilingual pivot translation. 1 Introduction Multilingual neural network models are models in which multiple languages are learned in a single model, and are useful for machine translation and cross-lingual language processing. Multilingual models utilize resources of similar languages (e.g., those in the same language family) and thus provide a relatively high degree of accuracy for even low-resource languages.1 Machine translation is performed according to a combination of a source language and a target language, and therefore, language-specific models require a model for each possible combina- tion of languages. By contrast, a multilingual model can handle all combinations of source and target languages and is therefore easier to manage. The potential usefulness of the multilingual model has led to the development of several encoder–decoder models pretrained using parallel corpora. For example, a multilingual translation model pretrained with the OPUS-100 corpus (Zhang et al., 2020)2 has been developed. This is a multilingual model that translates between English and any of 100 languages (i.e., an English-centric model). The M2M-100 model (Fan 1Under high-resource conditions, language-specific models are generally more accurate than multilingual models. This is called the curse of multilinguality. 2https://github.com/bzhangGo/zero/tree/master/docs/multilingual_laln_ lalt#pretrained-multilingual-models-many-to-many 349 Direct Translation Pivot Translation Ja :Bn Zh Ja :Bn Zh Ja :Bn Zh En Ja :Bn Zh Figure 1: Direct translation and pivot translation. et al., 2020)3 also handles 100 languages. It is pretrained in 2,200 directions by adding parallel corpora that do not include English. The NLLB-200 models (Team et al., 2022)4, which are extended from the M2M-100 model, handle around 200 languages and pretrained from parallel corpora of over 2,600 language pairs. Although the mBART models (Liu et al., 2020; Lewis et al., 2020) are also encoder–decoder pretrained models, they are trained using monolingual corpora only. If we handle 100 languages for translation, this results in a total of 9,900 translation direc- tions. Even if a multilingual translation model is used, translation quality for …

Content truncated to 500 words