Pre-Translation for Neural Machine Translation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1828–1836, Osaka, Japan, December 11-17 2016. Pre-Translation for Neural Machine Translation Jan Niehues, Eunah Cho, Thanh-Le Ha and Alex Waibel Institute for Anthropomatics Karlsruhe Institute of Technology, Germany firstname.surname@kit.edu Abstract Recently, the development of neural machine translation (NMT) has significantly improved the translation quality of automatic machine translation. While most sentences are more accurate and fluent than translations by statistical machine translation (SMT)-based systems, in some cases, the NMT system produces translations that have a completely different meaning. This is especially the case when rare words occur. When using statistical machine translation, it has already been shown that significant gains can be achieved by simplifying the input in a preprocessing step. A commonly used example is the pre-reordering approach. In this work, we used phrase-based machine translation to pre-translate the input into the target language. Then a neural machine translation system generates the final hypothesis using the pre-translation. Thereby, we use either only the output of the phrase-based machine translation (PBMT) system or a combination of the PBMT output and the source sentence. We evaluate the technique on the English to German translation task. Using this approach we are able to outperform the PBMT system as well as the baseline neural MT system by up to 2 BLEU points. We analyzed the influence of the quality of the initial system on the final result. 1 Introduction In the last years, statistical machine translation (SMT) system generated state-of-the-art performance for most language pairs. Recently, systems using neural machine translation (NMT) were able to outper- form SMT systems in several evaluations. These models are able to generate more fluent and accurate translation for most of sentences. Neural machine translation systems provide the output with high fluency. A weakness of NMT sys- tems, however, is that they sometimes lose the original meaning of the source words during translation. 1.One example from the first conference on machine translation (WMT16) test set is the segment in Table The English word goalie is not translated to the correct German word Torwart, but to the German word Gott, which means god. One problem could be that we need to limit the vocabulary size in order to train the model efficiently. We used Byte Pair Encoding (BPE) (Sennrich et al., 2016) to represent the text using a fixed size vocabulary. In our case the word goali is splitted into three parts go, al and ie. Then it is more difficult to transport the meaning to the translation. In contrast to this, in phrase-based machine translation (PBMT), we do not need to limit the vocabulary and are often able to translate words even if we have seen them only very rarely in the training. In the example mentioned before, for instance, the PBMT system had no problems translating the expression correctly. On the other hand, official evaluation campaigns (Bojar et al., 2016) have shown that NMT system often create grammatically correct sentence and are able …

Content truncated to 500 words