Findings of the Word-Level AutoCompletion Shared Task in WMT 2022

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 812 - 820 December 7-8, 2022 ©2022 Association for Computational Linguistics Findings of the Word-Level AutoCompletion Shared Task in WMT 2022∗ Francisco Casacuberta 1 George Foster 2 Guoping Huang 3 Philipp Koehn 4,5 Geza Kovacs 6 Lemao Liu 3 Shuming Shi 3 Taro Watanabe 7 Chengqing Zong 8 1 Universitat Politècnica de València 2 Google 3 Tencent AI Lab 4 Johns Hopkins University 5 Meta AI 6 LILT 7 Nara Institute of Science and Technology 8 Institute of Automation, Chinese Academy of Sciences Abstract Recent years have witnessed rapid advance- ments in machine translation, but the state-of- the-art machine translation system still can not satisfy the high requirements in some rigorous translation scenarios. Computer-aided transla- tion (CAT) provides a promising solution to yield a high-quality translation with a guaran- tee. Unfortunately, due to the lack of popular benchmarks, the research on CAT is not well developed compared with machine translation. In this year, we hold a new shared task called Word-level AutoCompletion (WLAC) for CAT in WMT. Specifically, we introduce some re- sources to train a WLAC model, and partic- ularly we collect data from CAT systems as a part of test data for this shared task. In ad- dition, we employ both automatic and human evaluations to measure the performance of the submitted systems, and our final evaluation re- sults reveal some findings for the WLAC task. 1 Introduction In past decades, the machine translation community has witnessed a significant evolution from statisti- cal machine translation (Koehn et al., 2003; Chi- ang, 2005; Koehn, 2009b) to neural machine trans- lation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2015; Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017). NMT has achieved a rapid and tremendous advancement in translation perfor- mance (Barrault et al., 2019). Despite its success in many real-world applications, its translation quality still can not satisfy the high requirements in some scenarios. In such rigorous scenarios, one promis- ing approach is to leverage machines to assist hu- man translation, such as Computer-aided Transla- tion (CAT) (Bowker, 2002; Koehn, 2009a; Foster et al., 1997; Langlais et al., 2000; Barrachina et al., 2009; Alabau et al., 2014; Knowles and Koehn, 2016; Santy et al., 2019). ∗ The authors are listed alphabetically. However, the development in CAT is much slower than in machine translation. For example, there are hundreds of research papers on machine translation in natural language processing confer- ences each year, whereas only a few papers on CAT are published. One of the main reasons is that there are few popular benchmarks or shared tasks for CAT research, which enable researchers to make continuous progress in this area. Consequently, in WMT this year, we hold a new shared task, Word- level AutoCompletion (WLAC), to facilitate the research in CAT. Generally, WLAC aims to auto- complete a word when a human translator types a sequence of characters (Huang et al., 2015; Li et al., 2021). As …

Content truncated to 500 words