FBK’s Neural Machine Translation Systems for IWSLT 2016

FBK’s Neural Machine Translation Systems for IWSLT 2016 M. Amin Farajian1,2, Rajen Chatterjee1,2, Costanza Conforti3, Shahab Jalalvand1,2, Vevake Balaraman2, Mattia A. Di Gangi1,2, Duygu Ataman1,2, Marco Turchi1, Matteo Negri1, Marcello Federico1 1Fondazione Bruno Kessler, Trento, Italy 2ICT Doctoral School, University of Trento, Italy 3Ludwig-Maximilian University of Munich, Germany federico@fbk.eu Abstract In this paper, we describe FBK’s neural machine transla- tion (NMT) systems submitted at the International Workshop on Spoken Language Translation (IWSLT) 2016. The sys- tems are based on the state-of-the-art NMT architecture that is equipped with a bi-directional encoder and an attention mechanism in the decoder. They leverage linguistic infor- mation such as lemmas and part-of-speech tags of the source words in the form of additional factors along with the words. We compare performances of word and subword NMT sys- tems along with different optimizers. Further, we explore different ensemble techniques to leverage multiple models within the same and across different networks. Several re- ranking methods are also explored. Our submissions cover all directions of the MSLT task, as well as en-{de, fr} and {de, fr}-en directions of TED. Compared to previously pub- lished best results on the TED 2014 test set, our models achieve comparable results on en-de and surpass them on en- fr (+2 BLEU) and fr-en (+7.7 BLEU) language pairs. 1. Introduction This paper reports on our first participation at IWSLT with neural machine translation (NMT) systems. This activity was carried out during a Summer project involving both members of FBK’s HLT-MT research unit and internship students.1 The main goal of the project was to gain knowledge on the Theano library and the sequence-to-sequence translation model by [1], also implemented in the Nematus toolkit [2]. During the project we explored different alternatives related to data selection, pre-/post-processing of the network’s in- put/output data, training modalities of the network and, fi- nally, decoding and re-scoring methods applied during the inference step. In this paper, we summarise our experience by reporting both on approaches that showed to be benefi- cial to the system performance, as well as on methods that apparently did not work as we expected. 1The first two authors contributed equally. The following authors con- tributed while attending a Summer internship with the HLT-MT research unit of FBK: Costanza Conforti, Ludwig-Maximilian University of Munich (Germany), Mattia di Gangi, University of Palermo (Italy), Vevake Balara- man, University of Trento (Italy). For IWSLT, we developed systems for the TED and the Microsoft Speech Language (MSLT) tasks of the MT track, in particular for the English-{German, French} and {German, French}-English directions. For each language pair we first developed generic systems that we trained on a mix of in-domain and out-domain data. Successively, generic systems were adapted to each task by using in-domain train- ing data. Out-domain training data was actually selected in order to improve training effectiveness both in terms of trans- lation quality and time. We employed word and factored models (lemma, POS) to represent the source text. We also trained a NMT network generating translations from right …

Content truncated to 500 words