Findings of the WMT 2022 Shared Task on Translation Suggestion

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 821 - 829 December 7-8, 2022 ©2022 Association for Computational Linguistics Findings of the WMT 2022 Shared Task on Translation Suggestion Zhen Yang, Fandong Meng, Yingxue Zhang, Ernan Li, and Jie Zhou Pattern Recognition Center, WeChat AI, Tencent Inc, Beijing, China {zieenyang, fandongmeng, yxuezhang, cardli, withtomzhou}@tencent.com Abstract We report the result of the first edition of the WMT shared task on Translation Suggestion (TS). The task aims to provide alternatives for specific words or phrases given the entire docu- ments generated by machine translation (MT). It consists two sub-tasks, namely, the naive translation suggestion and translation sugges- tion with hints. The main difference is that some hints are provided in sub-task two, there- fore, it is easier for the model to generate more accurate suggestions. For sub-task one, we provide the corpus for the language pairs English-German and English-Chinese. And only English-Chinese corpus is provided for the sub-task two. We received 92 submissions from 5 participat- ing teams in sub-task one and 6 submissions for the sub-task 2, most of them covering all of the translation directions. We used the automatic metric BLEU for evaluating the performance of each submission. 1 Introduction Computer-aided translation (CAT) (Barrachina et al., 2009; Green et al., 2014; Knowles and Koehn, 2016; Santy et al., 2019) has attained more and more attention for its promising ability in combin- ing the high efficiency of machine translation (MT) (Cho et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017) and high accuracy of human transla- tion (HT). A typical way for CAT tools to combine MT and HT is PE (Green et al., 2013; Zouhar et al., 2021), where the human translators are asked to provide alternatives for the incorrect word spans in the results generated by MT. To further reduce the post-editing time, researchers propose to apply TS into PE, where TS provides the sub-segment sug- gestions for the annotated incorrect word spans in the results of MT, and their extensive experiments show that TS can substantially reduce translators’ cognitive loads and the post-editing time (Wang et al., 2020; Lee et al., 2021). As there is no explicit and formal definition for TS, we observe that some previous works simi- lar or related to TS have been proposed (Alabau et al., 2014; Santy et al., 2019; Wang et al., 2020; Lee et al., 2021). However, there are two main pitfalls for these works in this line. First, most conventional works only focus on the overall per- formance of PE but ignore the exact performance of TS. This is mainly because the golden corpus for TS is relatively hard to collect. As TS is an im- portant sub-module in PE, paying more attention to the exact performance of TS can boost the perfor- mance and interpretability of PE. Second, almost all of the previous works conduct experiments on their in-house datasets or the noisy datasets built automatically, which makes their experiments hard to be followed and …

Content truncated to 500 words