A Template-based Method for Constrained Neural Machine Translation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3665 - 3679 December 7-11, 2022 ©2022 Association for Computational Linguistics A Template-based Method for Constrained Neural Machine Translation Shuo Wang1 Peng Li2∗ Zhixing Tan6 Zhaopeng Tu7 Maosong Sun1,4 Yang Liu1,2,3,4,5∗ 1Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China 1Beijing National Research Center for Information Science and Technology 2Institute for AI Industry Research, Tsinghua University, Beijing, China 3Beijing Academy of Artificial Intelligence, Beijing, China 4International Innovation Center of Tsinghua University, Shanghai, China 5Quan Cheng Laboratory 6Zhongguancun Laboratory, Beijing, P.R.China 7Tencent AI Lab Abstract Machine translation systems are expected to cope with various types of constraints in many practical scenarios. While neural machine translation (NMT) has achieved strong perfor- mance in unconstrained cases, it is non-trivial to impose pre-specified constraints into the translation process of NMT models. Although many approaches have been proposed to ad- dress this issue, most existing methods can not satisfy the following three desiderata at the same time: (1) high translation quality, (2) high match accuracy, and (3) low latency. In this work, we propose a template-based method that can yield results with high translation quality and match accuracy and the inference speed of our method is comparable with unconstrained NMT models. Our basic idea is to rearrange the generation of constrained and unconstrained to- kens through a template. Our method does not require any changes in the model architecture and the decoding algorithm. Experimental re- sults show that the proposed template-based approach can outperform several representative baselines in both lexically and structurally con- strained translation tasks. 1 1 Introduction Constrained machine translation is of important value for a wide range of practical applications, such as interactive translation with user-specified lexical constraints (Koehn, 2009; Li et al., 2020; Jon et al., 2021), domain adaptation with in-domain dictionaries (Michon et al., 2020; Niehues, 2021), and webpage translation with markup tags as struc- tural constraints (Hashimoto et al., 2019; Hanne- man and Dinu, 2020). Developing constrained neural machine translation (NMT) approaches can make NMT models applicable to more real-world scenarios (Bergmanis and Pinnis, 2021). ∗ Corresponding authors: P.Li (lipeng@air.tsinghua. edu.cn) and Y.Liu (liuyang2011@tsinghua.edu.cn). 1The source code is available at https://github.com/ THUNLP-MT/Template-NMT. However, it is challenging to directly impose constraints for NMT models due to their end-to- end nature (Post and Vilar, 2018). In accordance with this problem, a branch of studies modifies the decoding algorithm to take the constraints into ac- count when selecting candidates (Hokamp and Liu, 2017; Hasler et al., 2018; Post and Vilar, 2018; Hu et al., 2019; Hashimoto et al., 2019). Although constrained decoding algorithms can guarantee the presence of constrained tokens, they can signifi- cantly slow down the translation process (Wang et al., 2022) and can sometimes result in poor trans- lation quality (Zhang et al., 2021). Another branch of works constructs synthetic data to help NMT models acquire the ability to translate with constraints (Song et al., 2019; Dinu et al., 2019; Michon et al., …

Content truncated to 500 words