NJUNLP’s Participation for the WMT2022 Quality Estimation Shared Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 615 - 620 December 7-8, 2022 ©2022 Association for Computational Linguistics NJUNLP’s Participation for the WMT2022 Quality Estimation Shared Task Xiang Geng1, Yu Zhang1, Shujian Huang1∗, Shimin Tao2, Hao Yang2, Jiajun Chen1 1 National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China 2 Huawei Translation Services Center, Beijing, China {gx, zhangy}@smail.nju.edu.cn, {huangsj,chenjj}@nju.edu.cn {taoshimin, yanghao30}@huawei.com Abstract This paper presents submissions of the NJUNLP team in WMT 2022 Quality Estima- tion shared task 1, where the goal is to pre- dict the sentence-level and word-level quality for target machine translations. Our system explores pseudo data and multi-task learning. We propose several novel methods to gener- ate pseudo data for different annotations using the conditional masked language model and the neural machine translation model. The pro- posed methods control the decoding process to generate more real pseudo translations. We pre-train the XLMR-large model with pseudo data and then fine-tune this model with real data both in the way of multi-task learning. We jointly learn sentence-level scores (with regres- sion and rank tasks) and word-level tags (with a sequence tagging task). Our system obtains competitive results on different language pairs and ranks first place on both sentence- and word-level sub-tasks of the English-German language pair. 1 Introduction Quality Estimation (QE) of Machine Translation (MT) is a task to predict the quality of trans- lations at run-time without relying on reference translations (Specia et al., 2018). This paper de- scribes the contribution of the NJUNLP team to the WMT2022 QE Shared Task (Zerva et al., 2022) on sentence- and word-level sub-tasks (task 1)1. For the sentence-level task, participating systems are required to predict the quality score for each trans- lation output, and all scores are standardized using the z-score by the rater. The result is evaluated using Spearman’s rank correlation coefficient as the primary metric. For the word-level task, par- ticipating systems are required to tag each token of the translation output with OK and BAD. The ∗* Corresponding Author. 1https://wmt-qe-task.github.io/subtasks/ task1/ BAD tag denotes this token is wrong, or there is one or more missing token(s) on the left side. The result is evaluated in terms of Matthews correlation coefficient (MCC) as the primary metric. Inspired by DirectQE(Cui et al., 2021), we fur- ther explore pseudo data and multi-task learning for the QE shared task. Our main contributions are as follows: • We propose several novel methods to gener- ate pseudo data for different annotations using the conditional masked language model (Cui et al., 2021) and the neural machine transla- tion model (Vaswani et al., 2017). • We use the XLMR-large model (Conneau et al., 2020) as the QE model rather than a transformer base model with random initial- ization in (Cui et al., 2021). • We pre-train the QE model with pseudo data and then fine-tune it with real data both in the way of multi-task learning. We explore the rank task in addition to commonly used regression and sequence tagging …

Content truncated to 500 words