Training Neural Machine Translation to Apply Terminology Constraints

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3063–3068 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3063 Training Neural Machine Translation To Apply Terminology Constraints Georgiana Dinu Prashant Mathur Marcello Federico Yaser Al-Onaizan Amazon AI {gddinu,pramathu,marcfede,onaizan}@amazon.com Abstract This paper proposes a novel method to inject custom terminology into neural machine trans- lation at run time. Previous works have mainly proposed modifications to the decoding algo- rithm in order to constrain the output to in- clude run-time-provided target terms. While being effective, these constrained decoding methods add, however, significant computa- tional overhead to the inference step, and, as we show in this paper, can be brittle when tested in realistic conditions. In this paper we approach the problem by training a neural MT system to learn how to use custom terminol- ogy when provided with the input. Compara- tive experiments show that our method is not only more effective than a state-of-the-art im- plementation of constrained decoding, but is also as fast as constraint-free decoding. 1 Introduction Despite the high quality reached nowadays by neural machine translation (NMT), its output is of- ten still not adequate for many specific domains handled daily by the translation industry. While NMT has shown to benefit from the availability of in-domain parallel or monolingual data to learn domain specific terms (Farajian et al., 2018), it is not a universally applicable solution as often a do- main may be too narrow and lacking in data for such bootstrapping techniques to work. For this reason, most multilingual content providers main- tain terminologies for all their domains which are created by language specialists. For example, an entry such as Jaws (en) → Lo Squalo (it) would exist in order to indicate that the input Jaws is a scary movie should be translated as Lo Squalo `e un film pauroso. While translation memories can be seen as ready-to-use training data for NMT do- main adaptation, terminology databases (in short term bases) are more difficult to handle and there has been significant work on proposing methods to integrate domain terminology into NMT at run time. Constrained decoding is the main approach to this problem. In short, it uses the target side of ter- minology entries whose source side match the in- put as decoding-time constraints. Constrained de- coding and various improvements were addressed in Chatterjee et al. (2017), Hasler et al. (2018), Hokamp and Liu (2017) among others. Hokamp and Liu (2017) recently introduced the grid beam search (GBS) algorithm which uses a separate beam for each supplied lexical constraint. This so- lution however increases the run time complexity of the decoding process exponentially in the num- ber of constraints. Post and Vilar (2018) recently suggested using a dynamic beam allocation (DBA) technique that reduces the computational overhead to a constant factor, independent from the number of constraints. In practice, results reported in Post and Vilar (2018) show that constrained decoding with DBA is effective but still causes a …

Content truncated to 500 words