Tencent’s Multilingual Machine Translation System for WMT22 Large-Scale African Languages

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 1049 - 1056 December 7-8, 2022 ©2022 Association for Computational Linguistics Tencent’s Multilingual Machine Translation System for WMT22 Large-Scale African Languages Wenxiang Jiao Zhaopeng Tu Jiarui Li Wenxuan Wang Jen-tse Huang Shuming Shi Tencent AI Lab {joelwxjiao,zptu,ultrali,jwxwang,jentsehuang,shumingshi}@tencent.com Abstract This paper describes Tencent’s multilingual machine translation systems for the WMT22 shared task on Large-Scale Machine Transla- tion Evaluation for African Languages. We par- ticipated in the constrained translation track in which only the data and pretrained models provided by the organizer are allowed. The task is challenging due to three problems, in- cluding the absence of training data for some to-be-evaluated language pairs, the uneven op- timization of language pairs caused by data imbalance, and the curse of multilinguality. To address these problems, we adopt data augmen- tation, distributionally robust optimization, and language family grouping, respectively, to de- velop our multilingual neural machine transla- tion (MNMT) models. Our submissions won the 1st place on the blind test sets in terms of the automatic evaluation metrics.1 1 Introduction Multilingual neural machine translation (MNMT) aims to translate between multiple language pairs with a unified model (Johnson et al., 2017). It is appealing due to the model efficiency, easy de- ployment, and knowledge transfer between high resource languages and low resource languages. Hence, MNMT has attracted more and more atten- tion from both academia and industry. To improve the performance of MNMT models, previous re- searchers have proposed various approaches on advanced model architectures (Sen et al., 2019; Zhang et al., 2021), training strategies (Wang et al., 2020a,b), and data utilization (Siddhant et al., 2020; Wang et al., 2022). In addition, industrial compa- nies have released massive multilingual pretrained models (Tang et al., 2021) and large-scale multi- lingual translation models (Fan et al., 2021; Team 1Codes, models, and detailed competition results are available at https://github.com/wxjiao/ WMT2022-Large-Scale-African. et al., 2022) to facilitate translation among hun- dreds of languages. However, existing efforts on MNMT for African languages are not sufficient due to the lack of high quality and standardized evaluation benchmarks. In this paper, we build a system integrating several advanced approaches for WMT22 Large- Scale Machine Translation Evaluation Task (Ade- lani et al., 2022), which involves a set of 24 African languages. We participated in the Constrained Translation track, where only the data provided by the organizer are allowed. This task is challenging due to three potential problems: • The absence of training data for some to-be- evaluated language pairs; • The uneven optimization of language pairs due to data imbalance; • The curse of multilinguality in MNMT models caused by the hundreds of language pairs. For the first problem, we adopt data augmenta- tion techniques to construct synthetic data for the language pairs without parallel training data (§3.1). Specifically, we use back-translation (Sennrich et al., 2016) and self-training (Jiao et al., 2021), and attach a special tag to the synthetic side of the data. For the second issue, we utilize distribution- ally robust optimization …

Content truncated to 500 words