DUTNLP Machine Translation System for WMT22 General MT Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 397 - 402 December 7-8, 2022 ©2022 Association for Computational Linguistics DUTNLP Machine Translation System for WMT22 General MT Task Ting Wang Huan Liu Junpeng Liu Degen Huang∗ School of Computer Science, Dalian University of Technology {Wting_1513577,liuhuan4221,liujunpeng_nlp}@mail.dlut.edu.cn huangdg@dlut.edu.cn Abstract This paper describes DUTNLP Lab’s submis- sion to the WMT22 General MT Task on four translation directions: English to/from Chinese and English to/from Japanese under the con- strained condition. Our primary system are built on several Transformer variants which employ wider FFN layer or deeper encoder layer. The bilingual data are filtered by de- tailed data pre-processing strategies and four data augmentation methods are combined to en- large the training data with the provided mono- lingual data. Several common methods are also employed to further improve the model perfor- mance, such as fine-tuning, model ensemble and post-editing. As a result, our constrained systems achieve 29.01, 63.87, 41.84, and 24.82 BLEU scores on Chinese → English, English → Chinese, English → Japanese, and Japanese → English, respectively. 1 Introduction DUTNLP Lab participates in the WMT22 General MT Task on four translation directions: English ↔ Chinese and English ↔ Japanese. Our translation system is trained on the officially provided bilin- gual and monolingual data under the constrained condition. Several strategies such as fine-grained data pre-processing, large-scale synthetic data aug- mentation, diverse model architectures and domain fine-tuning are utilized to enhance the performance of the final ensemble model. Since the quality of the training data is crucial to the translation performance, all the training sets are filtered by the off-the-shelf toolkits and some manual rules. Details will be discussed in Section 2. Those data pre-processing strategies are also employed to filter out the synthetic data generated by different data augmentation methods. To generate synthetic parallel data, four data augmentation methods including back-translation ∗Corresponding author (Sennrich et al., 2016), forward-translation (Wu et al., 2019), knowledge distillation (Freitag et al., 2017) and R2L training (Liu et al., 2016) are employed in our experiments. Specifically, we leverage source-side monolingual data by explor- ing forward-translation, knowledge distillation and R2L training, while target-side monolingual data by back-translation. These strategies increase the data size to a large extent. The generated data and the original parallel data are combined to train NMT models. For model architectures, starting from Transformer-Big (Vaswani et al., 2017) settings, several Transformer variants are used to improve the model capacity and diversity. Previous studies (Bapna et al., 2018; Li et al., 2020) have shown that the translation performance can be significantly improved by increasing the model capacity. Therefore, we build different model architectures with either wider FFN layers (Ng et al., 2019) or deeper transformer encoder (Sun et al., 2019). Moreover, the Pre-Norm (Wang et al., 2019) is also adopted in all our experiments as its performance and training stability are better than the Post-Norm counterpart. Domain fine-tuning is the most effective method in our experiments, which greatly improves the translation performance. We first employ previous WMT …

Content truncated to 500 words