HW-TSC’s Participation at WMT 2021 Quality Estimation Shared Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 890–896 November 10–11, 2021. ©2021 Association for Computational Linguistics 890 HW-TSC’s Participation at WMT 2021 Quality Estimation Shared Task Yimeng Chen1∗, Chang Su1∗, Yingtao Zhang1∗, Yuxia Wang 1, Xiang Geng 2, Hao Yang 1, Shimin Tao 1, Jiaxin Guo 1, Minghan Wang 1, Min Zhang 1, Yujia Liu 1, Shujian Huang 2 1 Huawei Translation Services Center, Beijing, China 2 Nanjing University, Nanjing, China {chenyimeng,suchang8,zhangyintao9,wangyuxia5,yanghao30,taoshimin, guojiaxin1,wangminghan,zhangmin186,liuyujia13}@huawei.com {gx@smail, huangsj@}nju.edu.cn Abstract This paper presents our work in WMT 2021 Quality Estimation (QE) Shared Task. We par- ticipated in all of the three sub-tasks, including Sentence-Level Direct Assessment (DA) task, Word and Sentence-Level Post-editing Effort task and Critical Error Detection task, in all language pairs. Our systems employ the frame- work of Predictor-Estimator, concretely with a pre-trained XLM-Roberta as Predictor and task-specific classifier or regressor as Estima- tor. For all tasks, we improve our systems by incorporating post-edit sentence or additional high-quality translation sentence in the way of multitask learning or encoding it with pre- dictors directly. Moreover, in zero-shot set- ting, our data augmentation strategy based on Monte-Carlo Dropout brings up significant im- provement on DA sub-task. Notably, our sub- missions achieve remarkable results over all tasks. 1 Introduction Quality Estimation (QE) focuses on estimating the quality of machine translation (MT) system output when no ground truth reference is available (Specia et al., 2018). QE covers wide range of tasks includ- ing word-level, sentence-level and document-level. It has wide range of applications in MT quality check and post-editing effort estimation. In WMT2021 Quality Estimation shared task1, there are three sub tasks — Sentence-Level Direct Assessment task, Word and Sentence-Level Post- editing Effort task and Critical Error Detection task. Each sub task involves several language pairs. Our team participated in all the above three tasks over all language pairs. We summarized our main con- tributions as follow: • We employ Predictor-Estimator architecture (Kim et al., 2017b; Kim and Lee, 2016) which * Indicates equal contribution. 1http://www.statmt.org/wmt21/quality-estimation- task.html is a two-stage model consisting of a word pre- diction model trained from large-scale paral- lel corpora, and a estimation model trained from quality-annotated QE data. Different from the original Predictor-Estimator model in (Kim et al., 2017a), we use pre-trained XLM-Roberta large as predictor instead of RNN-based model to achieve better QE fea- tures, and use task-specific classifier or regres- sor as quality estimator. • We extend PE assisted QE (PEAQE) (Kepler et al., 2019; Wang et al., 2020) by integrating real PE or addtional high-quality translation in the way of multitask learning or directly encoding it with predictor. • We explore data augmentation method based on Monte Carlo (MC) dropout (Gal and Ghahramani, 2016) to enhance the perfor- mance of zero-shot language pairs in Direct Assessment(DA) task. Our methods achieve impressive performance on both word and sentence level tasks. Specifi- cally, we peak the top-1 on sentence-level DA over English-German and English-Japanese pairs. For word and sentence-level post-editing effort task, our submissions of the majority …

Content truncated to 500 words