Encouraging Neural Machine Translation to Satisfy Terminology Constraints

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1450–1455 August 1–6, 2021. ©2021 Association for Computational Linguistics 1450 Encouraging Neural Machine Translation to Satisfy Terminology Constraints Melissa Ailem, Jinghsu Liu and Raheel Qader Lingua Custodia, France {melissa.ailem,jingshu.liu,raheel.qader}@linguacustodia.com Abstract We present a new approach to encourage neu- ral machine translation to satisfy lexical con- straints. Our method acts at the training step and thereby avoiding the introduction of any extra computational overhead at inference step. The proposed method combines three main in- gredients. The first one consists in augment- ing the training data to specify the constraints. Intuitively, this encourages the model to learn a copy behavior when it encounters constraint terms. Compared to previous work, we use a simplified augmentation strategy without source factors. The second ingredient is con- straint token masking, which makes it even easier for the model to learn the copy behav- ior and generalize better. The third one, is a modification of the standard cross entropy loss to bias the model towards assigning high prob- abilities to constraint words. Empirical results show that our method improves upon related baselines in terms of both BLEU score and the percentage of generated constraint terms. 1 Introduction Neural Machine Translation (NMT) systems en- joy high performance and efficient inference (Sutskever et al., 2014; Bahdanau et al., 2014; Lu- ong et al., 2015; Vaswani et al., 2017). However, when it comes to domain specific scenarios, where it is often necessary to take into account terminol- ogy constraints, NMT models suffer from the lack of explicit source-target correspondences making it challenging to enforce such constraints. For in- stance, consider the following sentence from the fi- nancial domain : “Holders may submit instructions based on a minimum quantity being accepted by the offeror.”. According to the financial terminology, the words Holders and offeror should be translated porteurs and initiateur respectively. Unfortunately, a generic English-French NMT model would trans- late the above sentence as: “Les titulaires peuvent soumettre des instructions en fonction d’une quan- tit´e minimale accept´ee par l’offrant.”, where the words Holders and offeror are translated into tit- ulaires and offrant respectively. To address this limitation various approaches have been proposed. They can be grouped into two categories based on whether they enforce constraints at inference or at training time. The former family of methods changes the decoding step to inject the constraint terms in the output. While effective at satisfying constraints, these techniques tend to suffer from several weaknesses such as high computational cost at the decoding stage, decreased translation quality due to strict enforcement of terminology constraints (Hokamp and Liu, 2017; Post and Vilar, 2018), or ineptness if there are multiple constraints in the input/output (Susanto et al., 2020). The other category of methods, which we follow in this work, integrates lexical constraints during training (Dinu et al., 2019). More precisely, they augment the training data in such a way as to in- form the NMT model of the constrains that need to be satisfied (Crego et al., …

Content truncated to 500 words