The Mininglamp Machine Translation System for WMT21

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 260–264 November 10–11, 2021. ©2021 Association for Computational Linguistics 260 The Mininglamp Machine Translation System for WMT21 Shiyu Zhao, Xiaopu Li, Minghui Wu, Jie Hao Mininglamp Technology, Beijing, China {zhaoshiyu, lixiaopu, wuminghui, haojie}@mininglamp.com Abstract This paper describes Mininglamp neural ma- chine translation systems of the WMT2021 news translation tasks. We have participated in eight directions translation tasks for news text including Chinese↔English, Hausa↔English, German↔English and French↔German. Our fundamental system was based on Transformer architecture, with wider or smaller construc- tion for different news translation tasks. We mainly utilized the method of back-translation, knowledge distillation and fine-tuning to boost single model, while the ensemble was used to combine single models. Our final submission has ranked first for the English→Hausa task. 1 Introduction This paper describes the Mininglamp sub- missions to the WMT2021 news translation tasks for eight directions including four high- resource Chinese↔English, German↔English, two medium-resource French↔German and two low-resource Hausa↔English. Furthermore, all of our systems were built with constrained data sets. For this participation, we experimented with some smaller or wider Transformer (Vaswani et al., 2017) architectures to reach a reliable base- line based on different resource scales, sampling or beam search in back-translation to generate more suitable pseudo bilingual sentences. Particu- larly in the low-resource tasks, Hausa↔English, the Transformer-Small neural machine transla- tion was built for the baseline, we presented it- erative between back-translation and fine-tuning pattern which significantly improve the BLEU score on the validation set, and it worked well on English→Hausa task. Due to time constraints, we did not experiment on Hausa→English task. This path could be an experiment in the future work. As for the data augmentation aspect, we experi- mented with several back-translation methods (Sen- nrich et al., 2016a), including the beam search, un- restricted sampling and sampling-topK (Edunov et al., 2018), to leverage the target-side monolin- gual data. We also applied knowledge distillation (Freitag et al., 2017) to leverage the source-side monolingual data. Our systems followed four main steps:1) data filtering and preprocessing, 2) back-translation to generate pseudo bilingual data, 3) knowledge dis- tillation by monolingual data, 4) fine-tuning with in-domain. It should be emphasized that we used Mar- ian1 (Junczys-Dowmunt et al., 2018) to implement only for Hausa↔English baseline systems, and Fairseq2 (Ott et al., 2019) for the rest, include Hausa↔English back-translation and knowledge distillation models. 2 System Overview 2.1 Data Filtering and Preprocessing In this section, we discuss the preprocessing, nor- malization and filter techniques carried out in an attempt, in order to reduce spurious uncertainty in the modeling problem. 2.1.1 Text Preprocessing Generally, we carried out the following text prepro- cessing steps prior to use in every model: • Normalization: Unicode canonicalization, replacement of common multiple encoding errors present in training data, standardization of quotation marks into directional variants, conversion of any traditional Chinese char- acters into simplified forms, conversion of any Chinese full-width characters and seg- mental Chinese full-width punctuation into half-width forms. Normalize punctuation in all data by using …

Content truncated to 500 words