Lan-Bridge MT’s Participation in the WMT 2022 General Translation Shared Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 268 - 274 December 7-8, 2022 ©2022 Association for Computational Linguistics Lan-Bridge MT’s Participation in the WMT 2022 General Translation Shared Task Bing Han Lan-Bridge / Sichuan (China) hanbing@lan-bridge.com Yangjian Wu Lan-Bridge / Sichuan (China) wuyangjian@lan-bridge.com Gang Hu Lan-Bridge / Sichuan (China) hugang@lan-bridge.com Qiulin Chen Lan-Bridge / Sichuan (China) chenqiulin@lan-bridge.com Abstract This paper describes Lan-Bridge Translation systems for the WMT 2022 General Transla- tion shared task. We participate in 18 language directions: English to and from Czech, Ger- man, Ukrainian, Japanese, Russian, Chinese, English to Croatian, French to German, Yakut to and from Russian, and Ukrainian to and from Czech. We mainly focus on multilingual mod- els to develop systems covering all these direc- tions. In general, we apply data corpus filtering, scaling model size, sparse expert model (in par- ticular, Transformer with adapters), large-scale backtranslation, and language model reranking techniques. Our system ranks first in 6 direc- tions based on the automatic evaluation. 1 Introduction Our Lan-Bridge MT team participate in the WMT 2022 General Translation shared task. As machine translation expands into more and more languages, multilingual machine translation has attracted more and more attention in both academia and industry. It can not only avoid training a separate model for each language pair but also transfer knowledge from high-resource languages to low-resource ones. Many systems such as Tran et al. (2021) submit- ted in previous years have proved this point and achieved a state of the art results in some language directions. For data preprocessing, knowledge-based rules, language detection, and language model are in- volved to clean parallel data, monolingual data, and synthetic data (mainly from large-scale data mining and backtranslation). Punctuation normal- ization and BPE (byte pair encoding) (Sennrich et al., 2015) with subword regularization method (Provilkov et al., 2019) are applied for all lan- guages. As for models, we fork Fairseq (Ott et al., 2019) as our development tool and use Transformer (Vaswani et al., 2017) as the main architecture. In addition, we follow Bapna et al. (2019) to extend Transformer by adding language-specific adapters to bridge the gap between different language pairs. Finally, we ensemble dense Transformer models and sparse adapter models, and the final result are re-ranked by language models. For English to and from Chinese, we develop a separate system. In addition to optimization techniques similar to mul- tilingual models, We also use additional private data. And for Yakut to and from Russian, due to a smaller corpus, we simply apply fine-tuning and backtranslation on our multilingual models. We win the first place in Russian ↔ Yakut, Rus- sian → English, English → Croatian, Czech → English and Ukrainian → English based on BLEU (Papineni et al., 2002) score. 1 2 System Overview 2.1 Data Here we describe our base datasets, including bitext and monolingual data sources, and the preprocess- ing methods we apply to prepare these initial data sets to train our baseline models. 2.1.1 Bitext Data We use all available bitext …

Content truncated to 500 words