Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6675 - 6689 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding Soumya Chatterjee IIT Bombay soumya@cse.iitb.ac.in Sunita Sarawagi IIT Bombay sunita@iitb.ac.in Preethi Jyothi IIT Bombay pjyothi@cse.iitb.ac.in Abstract Online alignment in machine translation refers to the task of aligning a target word to a source word when the target sequence has only been partially decoded. Good online align- ments facilitate important applications such as lexically constrained translation where user- defined dictionaries are used to inject lexical constraints into the translation model. We propose a novel posterior alignment technique that is truly online in its execution and su- perior in terms of alignment error rates com- pared to existing methods. Our proposed in- ference technique jointly considers alignment and token probabilities in a principled man- ner and can be seamlessly integrated within existing constrained beam-search decoding al- gorithms. On five language pairs, including two distant language pairs, we achieve con- sistent drop in alignment error rates. When deployed on seven lexically constrained trans- lation tasks, we achieve significant improve- ments in BLEU specifically around the con- strained positions. 1 Introduction Online alignment seeks to align a target word to a source word at the decoding step when the word is output in an auto-regressive neural translation model (Kalchbrenner and Blunsom, 2013; Cho et al., 2014; Sutskever et al., 2014). This is un- like the more popular offline alignment task that uses the entire target sentence (Och and Ney, 2003). State of the art methods of offline alignment based on matching of whole source and target sentences (Jalili Sabet et al., 2020; Dou and Neubig, 2021) are not applicable for online alignment where we need to commit on the alignment of a target word based on only the generated prefix thus far. An important application of online alignment is lexically constrained translation which allows injection of domain-specific terminology and other phrasal constraints during decoding (Hasler et al., 2018; Hokamp and Liu, 2017; Alkhouli et al., 2018; Crego et al., 2016). Other applications include preservation of markups between the source and target (Müller, 2017), and supporting source word edits in summarization (Shen et al., 2019). These applications need to infer the specific source token which aligns with output token. Thus, alignment and translation is to be done simultaneously. Existing online alignment methods can be cate- gorized into Prior and Posterior alignment methods. Prior alignment methods (Garg et al., 2019; Song et al., 2020) extract alignment based on the atten- tion at time step t when outputting token yt. The at- tention probabilities at time-step t are conditioned on tokens output before time t. Thus, the alignment is estimated prior to observing yt. Naturally, the quality of alignment can be improved if we condi- tion on the target token yt (Shankar and Sarawagi, 2019). This motivated Chen et al. (2020) to propose a posterior alignment method where alignment is calculated …

Content truncated to 500 words