The TALP-UPC Neural Machine Translation System for German/Finnish-English Using the Inverse Direction Model in Rescoring

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 283–287 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics The TALP-UPC Neural Machine Translation System for German/Finnish-English Using the Inverse Direction Model in Rescoring Carlos Escolano, Marta R. Costa-juss`a and Jos´e A. R. Fonollosa TALP Research Center, Universitat Polit`ecnica de Catalunya, Barcelona carlos.escolano@tsc.upc.edu, {marta.ruiz,jose.fonollosa}@upc.edu Abstract In this paper, we describe the TALP- UPC participation in the News Task for German-English and Finish-English. Our primary submission implements a fully character to character neural machine translation architecture with an additional rescoring of a n-best list of hypothesis us- ing a forced back-translation to the source sentence. This model gives consistent im- provements on different pairs of languages for the language direction with the low- est performance while keeping the qual- ity in the direction with the highest perfor- mance. Additional experiments are reported for multilingual character to character neural machine translation, phrase-based trans- lation and the additional Turkish-English language pair. 1 Introduction Neural Machine Translation (MT) has been proven to reach state-of-the-art results in the last couple of years. The baseline encoder-decoder architecture has been improved by an attention- based mechanism citebahdanau:2015, subword units (Sennrich et al., 2016b), character-based encoders (Costa-juss`a and Fonollosa, 2016) or even with generative adversarial nets (Yang et al., 2017), among many others. Despite its successful beginnings, the neural MT approach still has many challenges to solve and improvements to incorporate into the system. However, since the system is computationally ex- pensive and training models may last for several weeks, it is not feasible to conduct multiple exper- iments for a mid-sized laboratory. For the same reason, it is also relevant to report negative results on NMT. In this system description, we describe our par- ticipation on German-English and Finnish-English for the News Task. Our system is a fully character- to-character neural MT (Lee et al., 2016) system with additional rescoring from the inverse direc- tion model. In parallel to our final system, we also experimented with multilingual character-to- character system using German, Finnish and Turk- ish on the source side and English on the target side. Unfortunately, these last experiments did not work. All our systems are contrasted with a standard phrase-based system built with Moses (Koehn et al., 2007). 2 Char-to-char Neural MT Our system uses the architecture from (Lee et al., 2016) where a character-level neural MT model maps the source character sequence to the target character sequence. The main difference in the encoder architecture respect to the standard neu- ral MT model from (Bahdanau et al., 2015) is the use of a segmentation-free fully character-level network that extends initial character-based ap- proaches like (Kim et al., 2015; Costa-juss`a and Fonollosa, 2016). In the encoder, the network architecture includes character embeddings, con- volution layers, max pooling and highway lay- ers. The resulting character-based representation is then used as input to a bidirectional recurrent neural network. The main difference in the de- coder architecture is that the single-layer feedfor- ward …

Content truncated to 500 words