Priming Neural Machine Translation

Proceedings of the 5th Conference on Machine Translation (WMT), pages 516–527 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 516 Priming Neural Machine Translation MinhQuang Pham†‡, Jitao Xu†‡, Josep Crego†, Jean Senellart†, Franc¸ois Yvon‡ †SYSTRAN, 5 rue Feydeau, 75002 Paris, France firstname.lastname@systrangroup.com ‡Universit´e Paris-Saclay, CNRS, LIMSI, 91400, Orsay, France firstname.lastname@limsi.fr Abstract Priming is a well known and studied psychol- ogy phenomenon based on the prior presen- tation of one stimulus (cue) to influence the processing of a response. In this paper, we propose a framework to mimic the process of priming in the context of neural machine translation (NMT). We evaluate the effect of using similar translations as priming cues on the NMT network. We propose a method to inject priming cues into the NMT network and compare our framework to other mecha- nisms that perform micro-adaptation during in- ference. Overall, experiments conducted in a multi-domain setting confirm that adding prim- ing cues in the NMT decoder can go a long way towards improving the translation accu- racy. Besides, we show the suitability of our framework to gather valuable information for an NMT network from monolingual resources. 1 Introduction Priming is a well studied human cognitive phe- nomenon, founded on the establishment of associa- tions between a stimulus and a response (Tulving et al., 1982). Multiple studies have shown how ex- ternal stimuli (cues) may have a profound effect on perception. In the case of language translation, ex- ternal stimuli having such effects are said to prime language understanding and potentially have a im- pact the actions of a human translator. Imagine for instance a translator facing the ambiguous sentence I was in the bank, and the effect on translation ac- curacy if primed with the cue river. Most likely, the human translator would consider the “edge of river” sense rather than “financial institution” for translation. In the context of human translation, cross-lingual priming is particularly effective as cues in the target language may notably influence the final translation word choice. Several research works have introduced the prim- ing analogy in deep neural networks. In computer vision priming has been broadly studied: for in- stance, in Rosenfeld et al. (2018), the authors intro- duce a cue about the presence of a certain class of object in an image that significantly improves ob- ject detection performance. Concerning language generation, Brown et al. (2020) use a combination of prompt and example to guide the GPT-3 network when performing a task, where the prompt is a sen- tence that describes the task (i.e. “Translate from English to French”); and is followed by an exam- ple of the task (i.e. “sea otter ; loutre de mer”). In the context of NMT, experiments reported (Sen- nrich et al., 2016a; Kobus et al., 2017; Dinu et al., 2019) aim at influencing translation inference with respectively politeness, domain and terminology constraints. More related to our work, (Bulte and Tezcan, 2019; Xu et al., 2020) introduce a simple and elegant framework where similar translations (cues) are used …

Content truncated to 500 words