LIMSI@IWSLT’16: MT Track

The IWSLT 2016 Evaluation Campaign M. Cettolo(1) J. Niehues(2) S. St¨uker(2) L. Bentivogli(1) R. Cattoni(1) M. Federico(1) (1) FBK - Via Sommarive 18, 38123 Trento, Italy (2) KIT - Adenauerring 2, 76131 Karlsruhe, Germany Abstract The IWSLT 2016 Evaluation Campaign featured two tasks: the translation of talks and the translation of video confer- ence conversations. While the first task extends previously offered tasks with talks from a different source, the second task is completely new. For both tasks, three tracks were organised: automatic speech recognition (ASR), spoken lan- guage translation (SLT), and machine translation (MT). Main translation directions that were offered are English to/from German and English to French. Additionally, the MT track included English to/from Arabic and Czech, as well as French to English. We received this year run submissions from 11 research labs. All runs were evaluated with objec- tive metrics, while submissions for two of the MT talk tasks were also evaluated with human post-editing. Results of the human evaluation show improvements over the best submis- sions of last year. 1. Introduction We report here on the outcomes of the 2016 evaluation cam- paign organized by the International Workshop of Spoken Language Translation. The IWSLT workshop was started in 2004 [1] with the purpose of enabling the exchange of knowledge among researchers working on speech translation and creating an opportunity to develop and compare transla- tion systems on a common test bed. The evaluation campaign built on one of the outcomes of the C-STAR (Consortium for Speech Translation Advanced Research) project, namely the BTEC (Basic Travel Expression Corpus) multilingual spo- ken language corpus [2], which initially served as a primary source of evaluation. Since its beginning, translation tasks of increasing difficulty were offered and new data sets cov- ering a large number of language pairs were shared with the research community. In the thirteen editions organized from 2004 to 2016, the campaign attracted around 70 different par- ticipating teams from all over the world. Automatic spoken language translation is particularly challenging for a number of reasons. On one side, machine translation (MT) systems are required to deal with the spe- cific features of spoken language. With respect to written language, speech is structurally less complex, formal and fluent. It is also characterized by shorter sentences with a lower amount of rephrasing but a higher pronoun density [3]. On the other side, speech translation [4] requires the integra- tion of MT with automatic speech recognition, which brings with it the additional difficulty of translating content that may have been corrupted by speech recognition errors. Along the years, three main evaluation tracks were pro- gressively introduced, addressing all the core technologies involved in the spoken language translation task, namely: • Automatic speech recognition (ASR), i.e. the conver- sion of a speech signal into a transcript • Machine translation (MT), i.e. the translation of a pol- ished transcript into another language • Spoken language translation (SLT), i.e. the conversion and translation of a speech signal into a transcript in another …

Content truncated to 500 words