Vega-MT: The JD Explore Academy Machine Translation System for WMT22

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 411 - 422 December 7-8, 2022 ©2022 Association for Computational Linguistics Vega-MT: The JD Explore Academy Translation System for WMT22 Changtong Zanℜ,♭ , Keqin Peng♯ , Liang Dingℜ , Baopu Qiu♮, Boan Liu♢, Shwai He△ Qingyu Lu♡, Zheng Zhang♢, Chuang Liu♢, Weifeng Liu♭, Yibing Zhanℜ, Dacheng Taoℜ ℜJD Explore Academy, JD.com Inc. ♭China University of Petroleum (East China) ♯Beihang University ♮Nanjing University ♢Wuhan University △University of Electronic Science and Technology of China ♡Southeast University dingliang1@jd.com Abstract We describe the JD Explore Academy’s submission of the WMT 2022 shared task on general machine translation. We participated in all high-resource tracks and one medium- resource track, including Chinese↔English (Zh↔En), German↔English (De↔En), Czech↔English (Cs↔En), Russian↔English (Ru↔En), and Japanese↔English (Ja↔En). [Method] We push the limit of our previous work – bidirectional training (Ding et al., 2021d) for translation by scaling up two main factors, i.e. language pairs and model sizes, namely the Vega-MT system. As for language pairs, we scale the “bidirectional” up to the “multidirectional” settings, covering all par- ticipating languages, to exploit the common knowledge across languages, and transfer them to the downstream bilingual tasks. As for model sizes, we scale the Transformer-BIG up to the extremely large model that owns nearly 4.7 Billion parameters, to fully enhance the model capacity for our Vega-MT. Also, we adopt the data augmentation strategies, e.g. cycle translation (Ding and Tao, 2019) for monolingual data, and bidirectional self- training (Ding and Tao, 2021) for bilingual and monolingual data, to comprehensively exploit the bilingual and monolingual data. To adapt our Vega-MT to the general domain test set, generalization tuning is designed. [Results] Based on the official automatic scores* of constrained systems, in terms of the SACREBLEU (Post, 2018) shown in Figure 1, we got the 1st place in {Zh-En (33.5), En-Zh (49.7), De-En (33.7), En-De (37.8), Cs-En (54.9), En-Cs (41.4) and En-Ru (32.7)}, 2nd place in {Ru-En (45.1) and Ja-En (25.6)}, and 3rd place in {En-Ja(41.5)}, respectively; W.R.T the COMET (Rei et al., 2020), we got the Equal contribution. Work was done when Chang- tong and Keqin were interning at JD Explore Academy. *https://github.com/wmt-conference/ wmt22-news-systems/tree/main/scores 1st place in {Zh-En (45.1), En-Zh (61.7), De-En (58.0), En-De (63.2), Cs-En (74.7), Ru-En (64.9), En-Ru (69.6) and En-Ja (65.1)}, 2nd place in {En-Cs (95.3) and Ja-En (40.6)}, respectively. Models will be released to facilitate the MT community through GitHub† and OmniForce Platform‡. 1st! 1st! 1st! 1st! 1st! 1st! 1st! 2nd" 2nd" 3rd# Figure 1: Vega-MT achieves 7 state-of-the-art BLEU points out of 10 high-resource translation tasks among all constrained systems, and significantly outperforms the competitive Transformer-BIG baselines. 1 Introduction In this year’s WMT general translation task, our Vega-MT translation team participated in 10 shared tasks, including Chinese↔English (Zh↔En), German↔English (De↔En), Czech↔English (Cs↔En), Russian↔English (Ru↔En), and Japanese↔English (Ja↔En). We use the same model architectures, data strategies and corresponding techniques for all tasks. †https://github.com/JDEA-NLP/Vega-MT ‡OmniForce Platform will be launched by JD Explore Academy 411 We aim to leverage the cross-lingual knowl- edge through …

Content truncated to 500 words