Modeling Target-Side Inflection in Neural Machine Translation

Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 32–42 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Modeling Target-Side Inflection in Neural Machine Translation Aleˇs Tamchyna1,2 and Marion Weller-Di Marco1,3 and Alexander Fraser1 1LMU Munich, 2Memsource, 3University of Stuttgart ales.tamchyna@memsource.com dimarco@ims.uni-stuttgart.de fraser@cis.lmu.de Abstract NMT systems have problems with large vocabulary sizes. Byte-pair encoding (BPE) is a popular approach to solving this problem, but while BPE allows the system to generate any target-side word, it does not enable effective generalization over the rich vocabulary in morpholog- ically rich languages with strong inflec- tional phenomena. We introduce a sim- ple approach to overcome this problem by training a system to produce the lemma of a word and its morphologically rich POS tag, which is then followed by a deter- ministic generation step. We apply this strategy for English–Czech and English– German translation scenarios, obtaining improvements in both settings. We fur- thermore show that the improvement is not due to only adding explicit morphological information. 1 Introduction Neural machine translation (NMT) has recently become the new state of the art. Despite a large body of recent research, NMT still remains a rela- tively unexplored territory. In this work, we focus on one of these less stud- ied areas, namely target-side morphology. NMT systems typically produce outputs word-by-word and at each step, they evaluate the probability of all possible target words. When translating to morphologically rich languages, due to the large size of target-side vocabularies, NMT systems run into scalability issues and struggle with vocabu- lary coverage. Byte-pair encoding (BPE, Sennrich et al. (2016b)) is currently perhaps the most success- ful approach to addressing these problems. How- ever, while BPE allows the system to generate any target-side word (possibly as a concatenation of smaller segments), it does not enable effec- tive generalization over the many different surface forms possible for a single lemma, which had been shown to be useful in phrase-based SMT (Bojar and Kos, 2010). We see three main problems associated with rich target-side morphology in NMT: (i) NMT sys- tems have no explicit connection between differ- ent surface forms of a single target-side lexeme (lemma), leading to data sparsity, (ii) there is no explicit information about morphological features of target-side words, and (iii) NMT systems can- not systematically generate unseen surface forms of known lemmas: while the combination of sub- word segments obtained with BPE splitting can technically generate new forms, this is not a lin- guistically informed way to generate new words, and is furthermore restricted to “simple” concate- native word formation processes. We propose a simple two-step approach to achieve morphological generalization in NMT. In the first step, we use an encoder-decoder NMT system with attention and BPE (Bahdanau et al., 2014; Sennrich et al., 2016b) to generate a se- quence of interleaving morphological tags and lemmas. In the second step, we use a morpholog- ical generator to produce the final inflected out- put. This decomposition addresses all three of the problems …

Content truncated to 500 words