Understanding and Improving the Robustness of Terminology Constraints in Neural Machine Translation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6029–6042 July 9-14, 2023 ©2023 Association for Computational Linguistics Understanding and Improving the Robustness of Terminology Constraints in Neural Machine Translation Huaao Zhang1, Qiang Wang1,2, Bo Qin1, Zelin Shi1, Haibo Wang1, Ming Chen1∗ 1 RoyalFlush AI Research Institute, Hangzhou, China 2 Zhejiang University, Hangzhou, China {zhanghuaao, wangqiang3, qinbo, shizelin}@myhexin.com, chm@zju.edu.cn Abstract In this work, we study the robustness of two typical terminology translation methods: Place- holder (PH) and Code-Switch (CS), concern- ing (1) the number of constraints and (2) the target constraint length. We identify that ex- isting terminology constraint test sets, such as IATE, Wiktionary, and TICO, are blind to this issue due to oversimplified constraint settings. To solve it, we create a new challenging test set of English-German, increasing the average constraint count per sentence from 1.1∼1.7 to 6.1 and the length per target constraint from 1.1∼1.2 words to 3.4 words. Then we find that PH and CS methods degrade as the number of constraints increases, but they have comple- mentary strengths. Specifically, PH is better at retaining high constraint accuracy but lower translation quality as measured by BLEU and COMET scores. In contrast, CS has the oppo- site results. Based on these observations, we propose a simple but effective method com- bining the advantages of PH and CS. This ap- proach involves training a model like PH to pre- dict the term labels, and then during inference replacing those labels with target terminology text like CS, so that the subsequent generation is aware of the target term content. Extensive experimental results show that this approach can achieve high constraint accuracy and trans- lation quality simultaneously, regardless of the number or length of constraints.1 1 Introduction Although Neural Machine Translation (NMT) has achieved expressive performance improvement with the increase of model and data scale, it still struggles when involved in mismatched domains and rare entities (Koehn and Knowles, 2017). Ter- minology constraints (TC) is a popular solution that requires the model to generate the translation following the pre-provided terminology pairs and ∗Corresponding author. 1https://github.com/zhajiahe/RTT has been widely applied in commercial translation systems, such as Google, DeepL, etc. Perhaps the most popular approach for TC is learning the constraint-aware model through data augmentation (Song et al., 2019; Dinu et al., 2019; Ailem et al., 2021; Bergmanis and Pinnis, 2021).2 Early data augmentation is based on placeholder (PH). During training, PH methods replace the terminology terms in both source and target sen- tences with ordered labels (e.g., “T1”, “T2”), while the model predicts labels rather than the concrete terms at inference (Crego et al., 2016; Michon et al., 2020). The main drawback of PH methods is that the term labels lose the original semantic informa- tion, resulting in incoherent translation. Unlike PH methods, Code-Switch (CS) methods follow the standard model and generate term translations word by word by injecting target constraints in the source sequence (Song et al., 2019; Dinu et al., 2019; …

Content truncated to 500 words