HW-TSC’s Participation at WMT 2020 Quality Estimation Shared Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 1056–1061 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 1056 HW-TSC’s Participation at WMT 2020 Quality Estimation Shared Task Minghan Wang1, Hao Yang1, Hengchao Shang1, Daimeng Wei1, Jiaxin Guo1, Lizhi Lei1, Ying Qin1, Shimin Tao1, Shiliang Sun3, Yimeng Chen1, Liangyou Li2 1Huawei Translation Services Center, Beijing, China 2Huawei Noah’s Ark Lab, Hong Kong, China 3East China Normal University, Shanghai, China {wangminghan,yanghao30,shanghengchao,weidaimeng,guojiaxin1, leilizhi,qinying,taoshimin,chenyimeng,liliangyou}@huawei.com slsun@cs.ecnu.edu.cn Abstract This paper presents our work in the WMT 2020 Word and Sentence-Level Post-editing Effort Quality Estimation (QE) Shared Task. Our system follows standard Predictor- Estimator architecture, with a pre-trained Transformer as the Predictor, and specific classifiers and regressors as Estimators. We integrate Bottleneck Adapter Layers in the Predictor to improve the transfer learning efficiency and prevent from over-fitting. At the same time, we jointly train the word- and sentence-level tasks with a unified model with multitask learning. Pseudo-PE assisted QE (PEAQE) is proposed, resulting in significant improvements on the performance. Our submissions achieve competitive result in word/sentence-level sub-tasks for both of En-De/Zh language pairs. 1 Introduction Quality Estimation (QE) assesses the translation quality of machine translation (MT) system output when ground truth reference is not available (Spe- cia et al., 2013, 2018). For the word-level QE task, participants are required to tag tokens and gaps of the translation output from an unknown MT sys- tem with OK and BAD, as well as tokens in the source with the same tags. The result is measured by Matthews Correlation Coefficient (MCC). For the sentence-level task, participants are required to predict the Human Translation Error Rate (HTER) scores of the machine translation outputs and the result is evaluated in terms of the Pearson’s corre- lation coefficient. Our team participates in some of the sub-tasks in two language pairs (En-De and En-Zh) (Specia et al., 2020). With regard to the En-De track, at word-level, our model achieves the MCC score of 0.5828 on the target side, and 0.5234 on the source side; at sentence-level, our model ranks the first place with a Pearson R score of 0.7583. With regard to the En-Zh track, we only submit the target side of word-level sub-task, and achieves a MCC score of 0.5872. Our system is implemented with fairseq (Ott et al., 2019) (for En-De track) and THUMT (Zhang et al., 2017) (for En-Zh track). We extend the origi- nal Transformer (Vaswani et al., 2017) model to fit the QE task, and leverage transfer learning to fine- tune the model with the task specific dataset (Dai and Le, 2015; Howard and Ruder, 2018; Radford et al., 2018). The contributions of our work are as follows: • We follow the Predictor-Estimator (Kim and Lee, 2016; Kim et al., 2017; Wang et al., 2018; Li et al., 2018; Kepler et al., 2019) architecture and build a unified QE model based on the standard Transformer MT model. • Bottleneck Adapter Layers (Houlsby et al., 2019; Yang et al., 2020) are integrated into the model for efficient …

Content truncated to 500 words