Lingua Custodia’s Participation at the WMT 2021 Machine Translation Using Terminologies Shared Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 799–803 November 10–11, 2021. ©2021 Association for Computational Linguistics 799 Lingua Custodia’s participation at the WMT 2021 Machine Translation using Terminologies shared task Melissa Ailem, Jinghsu Liu and Raheel Qader Lingua Custodia, France {melissa.ailem,jingshu.liu,raheel.qader}@linguacustodia.com Abstract This paper describes Lingua Custodia’s sub- mission to the WMT21 shared task on ma- chine translation using terminologies. We consider three directions, namely English to French, Russian, and Chinese. We rely on a Transformer-based architecture as a build- ing block, and we explore a method which introduces two main changes to the standard procedure to handle terminologies. The first one consists in augmenting the training data in such a way as to encourage the model to learn a copy behavior when it encounters ter- minology constraint terms. The second change is constraint token masking, whose purpose is to ease copy behavior learning and to im- prove model generalization. Empirical results show that our method satisfies most terminol- ogy constraints while maintaining high transla- tion quality. 1 Introduction Neural-based architectures have become standard for Machine Translation (MT), they are efficient and offer state-of-the-art performance in many sce- narios (Vaswani et al., 2017). However, these models often trained on very large corpora turn out to be less adequate in domains that require very careful use of terminology. For instance, consider the following English sentence from a biomedical corpus "now for the fever you can take a tachipirina sweet" . The term "tachipirina sweet" refers to "paracétamol" in French. Un- fortunately, a generic English-French Neural MT (NMT) model would translate the above sentence as: "maintenant pour la fièvre tu peux prendre un tachipirina bonbon", where the term "tachipi- rina sweet" is translated "tachipirina bonbon". The goal of the WMT21 shared task on machine translation using terminology constraints is to ex- plore methods that can take into account terminol- ogy constraints, in order to improve MT models’ ac- curacy and consistency on specific domains. In the literature there are two main families of methods to take into account specific terminologies. One family incorporates terminology constraints at in- ference (Post and Vilar, 2018; Susanto et al., 2020). Members of this category can guarantee strict en- forcement of constraints, however this often comes at the cost of higher decoding time and decreased accuracy (Hokamp and Liu, 2017; Post and Vilar, 2018). The other family of method integrates termi- nologies at training time (Dinu et al., 2019; Ailem et al., 2021), and they have the benefit of not chang- ing the NMT model as well as of not incurring additional computational overheads at inference time (Crego et al., 2016; Song et al., 2019; Dinu et al., 2019). We participate in the following three directions: English to French, Russian, and Chinese, and the system we submit falls into the second family of method incorporating terminologies at training time. More precisely, we explore a variant of the models proposed in (Ailem et al., 2021), which we train for each language pair. Following this work, …

Content truncated to 500 words