Document-Level Information as Side Constraints for Improved Neural Patent Translation

Document Information as Side Constraints for Improved Neural Patent Translation Laura Jehl jehl@cl.uni-heidelberg.de Computational Linguistics, Heidelberg University, Germany Stefan Riezler riezler@cl.uni-heidelberg.de Computational Linguistics & IWR, Heidelberg University, Germany Abstract We investigate the usefulness of document information as side constraints for machine trans- lation. We adapt two approaches to encoding this information as features for neural patent translation: As special tokens which are attached to the source sentence, and as tags which are attached to the source words. We found that sentence-attached features produced the same or better results as word-attached features. Both approaches produced significant gains of over 1% BLEU over the baseline on a German-English translation task, while sentence-attached features also produced significant gains of 0.7% BLEU on a Japanese-English task. We also describe a method to encode document information as additional phrase features for phrase- based translation, but did not find any improvements. 1 Introduction Document information beyond the text is readily available in many data sets, but is rarely used when building translation systems. Such information could comprise the document’s origin (time, place, author), its topic, or its connections to other documents. It exists, for example, in patents, textual content on the web, or e-commerce data. We aim to investigate the usefulness of document information as side constraints for translation. We use the term side constraints as it is used in Sennrich et al. (2016a) to mean information that is not present in a source string, but can influence translation choice in the target string.1 For example, patents are assigned to a hierarchical classification system indicating their topic(s) in various degrees of granularity. Depending on the topic, different translation choices may be required. The correct choice will not always be apparent from the sentence context. By providing the classification information to the translation model, we enable the model to select the correct translation, given the constraints. In this paper, we focus on patent translation. Patent translation lends itself particularly well to our endeavor since patent documents are annotated with different types of information, from hierarchical categorization to information about individual inventors. We are interested in seeing whether patent translation can be improved by this information and if so, which kind of information and which model integration is most useful. Since 2014, neural machine translation (NMT) has become the state of the art in ma- chine translation (Luong and Manning, 2015; Jean et al., 2015). We hypothesize that NMT is well-suited to the integration of document information. Since the model works on sentence rep- resentations, it can decide whether or not to pay attention to this information for each particular 1Sennrich et al. (2016a) explore politeness as a side constraint for translation. Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 1 translation in context. What is more, annotations could be highly correlated. For example, a patent document’s subclass label contains its class label. Deep models are capable of learning these correlations. Based on work by Kobus et al. (2016), …

Content truncated to 500 words