Guiding Neural Machine Translation Decoding with External Knowledge

Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 157–168 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Guiding Neural Machine Translation Decoding with External Knowledge Rajen Chatterjee(1,2), Matteo Negri(2), Marco Turchi(2), Marcello Federico(2), Lucia Specia(3), and Frédéric Blain(3) (1)University of Trento, Trento, Italy (2)Fondazione Bruno Kessler, Trento, Italy (3)University of Sheffield, Sheffield, UK {chatterjee,negri,turchi,federico}@fbk.eu {l.specia,f.blain}@sheffield.ac.uk Abstract Differently from the phrase-based paradigm, neural machine translation (NMT) operates on word and sentence representations in a continuous space. This makes the decoding process not only more difficult to interpret, but also harder to influence with external knowledge. For the latter problem, effective solutions like the XML-markup used by phrase-based models to inject fixed translation options as constraints at decoding time are not yet available. We propose a “guide” mechanism that enhances an existing NMT decoder with the ability to prioritize and adequately handle translation options presented in the form of XML annotations of source words. Positive results obtained in two different translation tasks indicate the effectiveness of our approach. 1 Introduction The need to enforce fixed translations of certain source words is a well known problem in machine translation (MT). For instance, this is an issue in application scenarios in which the translation process has to comply with specific terminology and/or style guides. In such situations it is gen- erally necessary to consider external resources to guide the decoder in order to ensure consistency or meet other specific requirements. Terminology lists, which provide the decoder with the expected translations of specific words or phrases, are a typ- ical example of external knowledge used to guide the process to meet such constraints. Meeting pre- defined constraints, however, does not represent the only case in which an external guidance can support decoding. In ensemble MT architectures, for example, the output of a translation system specialised in handling specific phenomena (e.g. numbers or dates) can be used to guide another decoder without changing its underlying model. Phrase-based statistical MT (PBSMT), which explicitly manipulates symbolic representations of the basic constituents (phrases) in the source and target languages, provides solutions to address these needs. For instance, the XML markup im- plemented in the Moses toolkit (Koehn et al., 2007) allows one to supply the expected transla- tions to the decoder in the form of tags surround- ing the corresponding source phrases. To our knowledge, solutions to this problem are not yet available for neural machine translation (NMT), which has recently emerged as the dom- inant approach for MT. In particular, no work has been done to address the needs of the translation industry, in which language service providers usu- ally receive translation requests that must be sat- isfied in short time, often taking into account ex- ternal knowledge that defines specific customers’ constraints. In this case, the time-consuming re- training routines of NMT are not viable, thus mak- ing methods to inject external knowledge without retraining of paramount importance. To address this gap, we investigate problems arising from the fact that …

Content truncated to 500 words