Towards Making the Most of Pre-trained Translation Model for Quality Estimation

Towards Making the Most of Pre-trained Translation Model for Quality Estimation Chunyou Li1, Hui Di2, Hui Huang1, Kazushige Ouchi2 Yufeng Chen1, Jian Liu1, Jinan Xu1∗ 1 Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing 100044, China 2 Toshiba (China) Co., Ltd. {21120368,chenyf,jianliu,jaxu}@bjtu.edu.cn; dihui@toshiba.com.cn; huanghui hit@126.com; kazushige.ouchi@toshiba.co.jp Abstract Machine translation quality estimation (QE) aims to evaluate the quality of machine translation automatically without relying on any reference. One common practice is applying the translation model as a feature extractor. However, there exist several discrepancies between the transla- tion model and the QE model. The translation model is trained in an autoregressive manner, while the QE model is performed in a non-autoregressive manner. Besides, the translation model only learns to model human-crafted parallel data, while the QE model needs to model machine- translated noisy data. In order to bridge these discrepancies, we propose two strategies to post- train the translation model, namely Conditional Masked Language Modeling (CMLM) and De- noising Restoration (DR). Specifically, CMLM learns to predict masked tokens at the target side conditioned on the source sentence. DR firstly introduces noise to the target side of parallel data, and the model is trained to detect and recover the introduced noise. Both strategies can adapt the pre-trained translation model to the QE-style prediction task. Experimental results show that our model achieves impressive results, significantly outperforming the baseline model, verifying the effectiveness of our proposed methods. 1 Introduction Machine translation has always been the hotspot and focus of research. Compared with traditional meth- ods, neural machine translation (NMT) has achieved great success. However, current translation systems are still not perfect to meet the real-world applications without human post-editing. Therefore, to carry out risk assessment and quality control for machine translation, how to evaluate the quality of machine translation is also an important problem. Quality Estimation (QE) aims to predict the quality of machine translation automatically without rely- ing on reference. Compared with commonly used machine translation metrics such as BLEU (Papineni et al., 2002) and METEOR (Lavie and Denkowski, 2009), QE can be applicable to the case where refer- ence translations are unavailable. It has a wide range of applications in post-editing and quality control for machine translation. The biggest challenge for QE is data scarcity. Since QE data is often limited in size, it is natural to transfer bilingual knowledge from parallel data to the QE task. One well-known framework for this knowledge transfer is the predictor-estimator framework, in which the predictor is trained on large parallel data and used to extract features, and the estimator will make quality estimation based on features provided by the predictor. The predictor is usually a machine trans- lation model, which can hopefully capture the alignment or semantic information of the source and the target in a pair. Kim et al. (2016) first proposed to use an RNN-based machine translation model as the feature extractor, to leverage massive parallel data to alleviate the sparsity of annotated QE data. Wang et al. …

Content truncated to 500 words