Multi-Domain Neural Machine Translation through Unsupervised Adaptation

Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 127–137 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Multi-Domain Neural Machine Translation through Unsupervised Adaptation M. Amin Farajian(1,2), Marco Turchi(1), Matteo Negri(1), Marcello Federico(1) (1)Fondazione Bruno Kessler, Trento, Italy (2)University of Trento, Trento, Italy {farajian,turchi,negri,federico}@fbk.eu Abstract We investigate the application of Neu- ral Machine Translation (NMT) under the following three conditions posed by real- world application scenarios. First, we op- erate with an input stream of sentences coming from many different domains and with no predefined order. Second, the sen- tences are presented without domain in- formation. Third, the input stream should be processed by a single generic NMT model. To tackle the weaknesses of cur- rent NMT technology in this unsupervised multi-domain setting, we explore an ef- ficient instance-based adaptation method that, by exploiting the similarity between the training instances and each test sen- tence, dynamically sets the hyperparame- ters of the learning algorithm and updates the generic model on-the-fly. The results of our experiments with multi-domain data show that local adaptation outperforms not only the original generic NMT system, but also a strong phrase-based system and even single-domain NMT models specifi- cally optimized on each domain and appli- cable only by violating two of our afore- mentioned assumptions. 1 Introduction The progress towards a more pervasive integra- tion of machine translation (MT) into industrial translation workflows has to confront two inter- connected problems. On one side, MT technology should be able to guarantee a high level of flexibil- ity to deliver good-quality output in a wide range of use scenarios (language combinations, genres, domains). On the other side, the infrastructures required to reach this objective should be scalable enough to enable the industrial deployment of MT at reasonable cost. The first problem is a well known one in (sta- tistical) MT: regardless of the paradigm adopted, performance is bounded by the similarity between training and test data. The scenario addressed in this paper, in which the input stream comes from a variety of different domains, is a typical example where models trained on generic parallel corpora suffer from data diversity. Indeed, processing sen- tences from diverse domains becomes more and more difficult when the distance from the train- ing instances increases. The more the domains a system is exposed to, the higher the chance to ex- perience drops in translation quality under unseen conditions. To cope with this issue, MT systems should be flexible enough to adapt to a variety of linguistic differences (e.g. lexical, structural) be- tween different data points. The second problem is more practical: in ab- sence of flexible models, multi-domain translation scenarios call for infrastructures based on multi- ple specialised systems, each of which is tuned to maximise performance in a given domain. This solution, however, has two evident drawbacks: i) domain-specific models can only be invoked by input sentences presented along with domain in- formation, so that each instance is processed by the right model, and ii) …

Content truncated to 500 words