Continuous Learning in Neural Machine Translation using Bilingual Dictionaries

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 830–840 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 830 Continuous Learning in Neural Machine Translation using Bilingual Dictionaries Jan Niehues Department of Data Science and Knowledge Engineering (DKE) Maastricht University, Maastricht, The Netherlands jan.niehues@maastrichtuniversity.nl Abstract While recent advances in deep learning led to significant improvements in machine transla- tion, neural machine translation is often still not able to continuously adapt to the envi- ronment. For humans, as well as for ma- chine translation, bilingual dictionaries are a promising knowledge source to continuously integrate new knowledge. However, their ex- ploitation poses several challenges: The sys- tem needs to be able to perform one-shot learn- ing as well as model the morphology of source and target language. In this work, we proposed an evaluation frame- work to assess the ability of neural machine translation to continuously learn new phrases. We integrate one-shot learning methods for neural machine translation with different word representations and show that it is important to address both in order to successfully make use of bilingual dictionaries. By addressing both challenges we are able to improve the ability to translate new, rare words and phrases from 30% to up to 70%. The correct lemma is even generated by more than 90%. 1 Introduction Recent advances in neural machine translation (NMT) have led to astonishing translation quality of research systems in evaluation campaigns as well as for commercial systems. These improvements even led to discussions whether automatic machine translation is already on par with human transla- tion (Barrault et al., 2019). One challenge that has raised less attention is the ability of these systems to continuously learn over time. In contrast, hu- mans are continuously improving their skills and adapting to an ever-changing environment. There are several reasons why this is necessary: First, nobody is fluent in all possible domains. Even professional translators need to adapt to the specific vocabulary of different domains. Secondly, language is not static but developing over time and translators need to learn new terms, meanings and expressions. For humans, one successful approach to adapt to the environment is the usage of a dictionary 1. Learning translations from a dictionary has several advantages: Dictionaries contain minimal exam- ples. We do not need to collect full sentences, but can directly learn translations from a single phrase. Furthermore, this can even be generalized to other inflected forms of the same lexem. Secondly, it en- ables the system to directly integrate correction. If a user sees a specific problem, the user can interact with the system by adding a specific dictionary en- try. This is very important if a specific terminology should be used. Motivated by the success for human translators, in this work we will enable NMT to also success- fully integrate knowledge from bilingual dictionar- ies. Thereby, we will focus on learning transla- tions that could not be learned from parallel data. This poses several interesting research challenges …

Content truncated to 500 words