Improving Character-Based Decoding Using Target-Side Morphological Information for Neural Machine Translation

Proceedings of NAACL-HLT 2018, pages 58–68 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Improving Character-based Decoding Using Target-Side Morphological Information for Neural Machine Translation Peyman Passban, Qun Liu, Andy Way ADAPT Centre School of Computing Dublin City University, Ireland firstname.lastname@adaptcentre.ie Abstract Recently, neural machine translation (NMT) has emerged as a powerful alterna- tive to conventional statistical approaches. However, its performance drops consider- ably in the presence of morphologically rich languages (MRLs). Neural engines usually fail to tackle the large vocabulary and high out-of-vocabulary (OOV) word rate of MRLs. Therefore, it is not suitable to exploit existing word-based models to translate this set of languages. In this paper, we propose an extension to the state-of-the-art model of Chung et al. (2016), which works at the character level and boosts the decoder with target-side morphological information. In our archi- tecture, an additional morphology table is plugged into the model. Each time the decoder samples from a target vocabulary, the table sends auxiliary signals from the most relevant affixes in order to enrich the decoder’s current state and constrain it to provide better predictions. We evaluated our model to translate English into Ger- man, Russian, and Turkish as three MRLs and observed significant improvements. 1 Introduction Morphologically complex words (MCWs) are multi-layer structures which consist of different subunits, each of which carries semantic informa- tion and has a specific syntactic role. Table 1 gives a Turkish example to show this type of complexity. This example is a clear indication that word-based models are not suitable to process such complex languages. Accordingly, when translating MRLs, it might not be a good idea to treat words as atomic units as it demands a large vocabulary that im- poses extra overhead. Since MCWs can appear in various forms we require a very large vocabu- lary to i) cover as many morphological forms and words as we can, and ii) reduce the number of OOVs. Neural models by their nature are com- plex, and we do not want to make them more com- plicated by working with large vocabularies. Fur- thermore, even if we have quite a large vocabulary set, clearly some words would remain uncovered by that. This means that a large vocabulary not only complicates the entire process, but also does not necessarily mitigate the OOV problem. For these reasons we propose an NMT engine which works at the character level. Word Translation terbiye good manners terbiye.siz rude terbiye.siz.lik rudeness terbiye.siz.lik.leri their rudeness terbiye.siz.lik.leri.nden from their rudeness Table 1: Illustrating subword units in MCWs. The boldfaced part indicates the stem. In this paper, we focus on translating into MRLs and issues associated with word formation on the target side. To provide a better translation we do not necessarily need a large target lexicon, as an MCW can be gradually formed during decod- ing by means of its subunits, similar to the solu- tion proposed in character-based decoding models (Chung et al., 2016). Generating a complex word character-by-character is a …

Content truncated to 500 words