Lexical Micro-adaptation for Neural Machine Translation

Lexical Micro-adaptation for Neural Machine Translation Jitao Xu, Josep Crego, Jean Senellart SYSTRAN 5 rue Feydeau, 75002 Paris (France) firstname.lastname@systrangroup.com Abstract This work is inspired by a typical machine translation in- dustry scenario in which translators make use of in-domain data for facilitating translation of similar or repeating sen- tences. We introduce a generic framework applied at infer- ence in which a subset of segment pairs are first extracted from training data according to their similarity to the input sentences. These segments are then used to dynamically up- date the parameters of a generic NMT network, thus perform- ing a lexical micro-adaptation. Our approach demonstrates strong adaptation performance to new and existing datasets including pseudo in-domain data. We evaluate our approach on a heterogeneous English-French training dataset showing accuracy gains on all evaluated domains when compared to strong adaptation baselines. 1. Introduction High-quality domain-specific translation is crucial for nowa- days machine translation industry, which has adopted neural machine translation (NMT) as its dominating paradigm [1, 2, 3, 4, 5]. The data-driven nature of NMT conditions the quality of translations to the availability of large volumes of adequate training resources for a given domain. Despite that an ever increasing amount of data is becoming available, domain-specific resources are usually scarce and the most common approach when dealing with domain specific data is either to train multi-domain models [6] and dynamically provide a token to select the domain or to fine-tune a generic translation model by running additional learning iterations (fine-tuning) over in-domain data [7]. However, these solu- tions involving prior training of models are a) expensive on industrial production workflows, given the large number of specialised models usually required, resulting from the com- bination of genres, domains, styles, customer products, etc, b) impracticable and not flexible-enough to account for fur- ther fine-tuning to specific translators, covering very specific datasets [8, 9]. This work covers a typical industry scenario where high- quality translations are needed of repetitive material for which similar examples are available in the training dataset (e.g. translation of product manuals, drafts of legislation, technical documentation, etc.). Texts to be translated are rel- atively large and built from semantically related sentences. In this context translations are usually human post-edited, con- sequently, making available a valuable knowledge that could be incorporated by MT engines to further boost translation of upcoming documents. We suggest a simple but yet powerful micro-adaptation framework applied on-the-fly at inference- time in two steps: first, a reduced set of training instances similar to input sentences are retrieved from the training dataset; second, such retrieved sentences together with their corresponding human translations are used to update the pa- rameters of a generic NMT network. These two steps being repeated for each new translation batch. A main drawback of our micro-adaptation framework is the additional workload when translating new documents. However, little overhead is introduced by sentence retrieval, which can be performed very efficiently, while adaptation can be speed up by reduc- ing the number of training examples it …

Content truncated to 500 words