OPPO’s Machine Translation Systems for WMT20

Proceedings of the 5th Conference on Machine Translation (WMT), pages 282–292 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 282 OPPO’s Machine Translation Systems for WMT20 Tingxun Shi, Shiyu Zhao, Xiaopu Li, Xiaoxue Wang, Qian Zhang Di Ai, Dawei Dang, Zhengshan Xue and Jie Hao Manifold Lab, OPPO Research Institute, Beijing, China {shitingxun, zhaoshiyu, lixiaopu, wangxiaoxue, zhangqian666, aidi1, dangdawei, xuezhengshan, haojie}@oppo.com Abstract In this paper we demonstrate our (OPPO’s) machine translation systems for the WMT20 Shared Task on News Translation for all the 22 language pairs. We will give an overview of the common aspects across all the systems firstly, including two parts: the data prepro­ cessing part will show how the data are pre­ processed and filtered, and the system part will show our models architecture and the tech­ niques we followed. Detailed information, such as training hyperparameters and the re­ sults generated by each technique will be de­ picted in the corresponding subsections. Our final submissions ranked top in 6 directions (English ↔ Czech, English ↔ Russian, French → German and Tamil → English), third in 2 directions (English → German, English → Japanese), and fourth in 2 directions (English → Pashto and and English → Tamil). 1 Introduction This paper describes the OPPO’s submission to the Fifth Conference on Machine Translation (WMT20) news translation shared task. We built Transformer (Vaswani et al., 2017)­based systems for all the directions, and applied several well­ known, widely­used techniques, such as large­ scale back­translation (Sennrich et al., 2016a) and forward­translation, model ensemble and rerank­ ing. Since all the systems share a roughly similar data preprocessing and training methods, to avoid duplication words, we will demonstrate the com­ mon knowledge in Section 2 firstly, which will be divided into two parts: the preprocessing part shows the data preprocessing pipeline and data fil­ tering pipeline, the latter is generally composed by rule­based filtering and alignment­based filtering; the training part depicts the techniques we applied. Detailed information, including training hyperpa­ rameters, the results generated by each technique, and some other explorations will be listed in each corresponding direction in Section 3. Finally, we will summarize the report and indicate our final works. We used marian (Junczys­Dowmunt et al., 2018) to implement our systems for English ↔ {Khmer, Russian, Tamil} and French ↔ German task pairs, and fairseq (Ott et al., 2019) for the rest 1. 2 System Overview We preprocess corpora in two stages. In the pre­ processing stage, data is converted but not filtered. The common pipeline of preprocessing including the following steps: • Remove non­utf8 characters • Unescape html characters, e.g. “>” is con­ verted to “>” • Normalize different kinds of spaces and punc­ tuations • Tokenization • True case The last three steps are all processed by moses scripts. This pipeline is both applied for the paral­ lel corpora and monolingual corpora, and true case models are generally trained on the mixture of par­ allel and monolingual datasets. After preprocessing we filter the parallel cor­ pora according to statistical …

Content truncated to 500 words