HW-TSC’s Participation at WMT 2020 Automatic Post Editing Shared Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 797–802 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 797 HW-TSC’s Participation at WMT 2020 Automatic Post Editing Shared Task Hao Yang1, Minghan Wang1, Daimeng Wei1, Hengchao Shang1, Jiaxin Guo1, Zongyao Li1, Lizhi Lei1, Ying Qin1, Shimin Tao1, Shiliang Sun2, Yimeng Chen1 1Huawei Translation Services Center, Beijing, China 2East China Normal University, Shanghai, China {yanghao30,wangminghan,weidaimeng,shanghengchao,guojiaxin1, lizongyao,leilizhi, qinying,taoshimin,chenyimeng}@huawei.com slsun@cs.ecnu.edu.cn Abstract The paper presents the submission by HW- TSC in the WMT 2020 Automatic Post Edit- ing Shared Task. We participate in the English→German and English→Chinese lan- guage pairs. Our system is built based on the Transformer pre-trained on WMT 2019 and WMT 2020 News Translation corpora, and fine-tuned on the APE corpus. Bottleneck Adapter Layers are integrated into the model to prevent over-fitting. We further collect ex- ternal translations as the augmented MT can- didates to improve the performance. The ex- periment demonstrates that pre-trained NMT models are effective when fine-tuning with the APE corpus of a limited size, and the perfor- mance can be further improved with external MT augmentation. Our system achieves com- petitive results on both directions in the final evaluation. 1 Introduction Automatic post editing (APE) has been used in many scenarios where the performance of a black- box Machine Translation (MT) system is unknown, or, domain specific corrections are required (Pal et al., 2016; Junczys-Dowmunt and Grundkiewicz, 2017; Correia and Martins, 2019; Chatterjee et al., 2020). The continuous improvements of NMT sys- tems’s performances along with deep learning ad- vancements insert great challenges on developing sound APE systems, as simple translation errors are rarely seen in machine translation outputs nowa- days while the remaining errors are still tough to solve. Transfer learning and data augmentation techniques have demonstrated their efficiency in re- cent years when models are trained on datasets with limited size (Devlin et al., 2018). Therefore, such techniques are also adopted in APE tasks (Lopes et al., 2019; Chatterjee et al., 2019). Participants in the APE tasks are required de- velop systems to automatically post edit the trans- lation outputs from an unknown MT system (Chat- terjee et al., 2019). In this year, the dataset has changed in terms of domain (from IT to Wikipedia) and quality of MT ( a significant decrease in BLEU). Using previous dataset or officially pro- vided synthetic corpus (such as Artificial and eSCAPE) (Junczys-Dowmunt and Grundkiewicz, 2016; Negri et al., 2018) to enlarge the training set might not be appropriate under such circumstance due to the change in data distribution. Therefore, we decide to perform transfer learning with the offi- cially released training set and integrate Bottleneck Adapter Layers (BAL) (Houlsby et al., 2019; Yang et al., 2020) to prevent over-fitting. Our model is built based on Transformer (Vaswani et al., 2017) and is pre-trained on the WMT 2019 and 2020 news translation corpora. Compared with the work by (Lopes et al., 2019), we consider that it is more intuitive to use a pre- trained NMT model …

Content truncated to 500 words