POLENG MT: An Adaptive MT Platform

Multi-Domain Adaptation in Neural Machine Translation with Dynamic Sampling Strategies Minh-Quang Pham Uni. Paris-Saclay, CNRS, LISN F-91405 Orsay pham@limsi.fr Josep Crego SYSTRAN , 5 rue Feydeau, F-75002 Paris crego@systrangroup.com Franc¸ois Yvon Uni. Paris-Saclay, CNRS, LISN F-91405 Orsay, France yvon@limsi.fr Abstract Building effective Neural Machine Trans- lation models often implies accommodat- ing diverse sets of heterogeneous data so as to optimize performance for the do- main(s) of interest. Such multi-source / multi-domain adaptation problems are typi- cally approached through instance selection or reweighting strategies, based on a static assessment of the relevance of training in- stances with respect to the task at hand. In this paper, we study dynamic data selec- tion strategies that are able to automatically re-evaluate the usefulness of data samples in the course of training. Based on the re- sults of multiple experiments, we show that our method offer a generic framework to automatically handle several real-world sit- uations, from multi-source or unsupervised domain adaptation to multidomain learning. 1 Introduction A typical setting in machine translation (MT) is to collect the largest possible collection of parallel data for the chosen language pair, with the intent to achieve optimal performance for the task of inter- est. In such situations, the training data distribution is opportunistic, while the test data distribution is chosen and fixed; a key aspect of training is then to mitigate the detrimental effects of a mismatch be- tween these distributions. Single-source and multi- source1 domain adaptation (DA) is a well-studied © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1In this paper, multi-source DA means having multiple do- mains to adapt from; this setting differs from multi-source translation, where several source languages are considered. instance of this setting (see (Chu et al., 2017; Saun- ders, 2021) for a review), and so is multi-domain (MD) learning (Chu and Dabre, 2018; Zeng et al., 2018; Jiang et al., 2020; Pham et al., 2021). A re- lated situation is multilingual MT (Firat et al., 2016; Ha et al., 2016; Johnson et al., 2017; Aharoni et al., 2019), where the diversity of training data not only corresponds to variations in the topic, genre, or register but also in language. This problem is often approached by static in- stance selection or re-weighting strategies, where the available training data is used in proportion to its relevance for the testing conditions (Moore and Lewis, 2010; Axelrod et al., 2011). Finding the optimal balance of training data is however, a challenging task due, for instance, to the similarity between domains/languages, or to the regulariza- tion effects of out-of-domain data (Miceli Barone et al., 2017). A static policy may also be suboptimal when some target domains or languages are easier to train than others. Finally, improving the perfor- mance of the MT system in one domain will often hurt that of another (van der Wees et al., 2017; Britz et al., 2017) and improving model generalization across all domains (Koehn et al., 2018) …

Content truncated to 500 words