The UMD Machine Translation Systems at IWSLT 2016: English-to-French Translation of Speech Transcripts

The UMD Machine Translation Systems at IWSLT 2016: English-to-French Translation of Speech Transcripts Xing Niu, Marine Carpuat Dept. of Computer Science University of Maryland, College Park {xingniu,marine}@cs.umd.edu Abstract We describe the University of Maryland machine translation system submitted to the IWSLT 2016 Microsoft Speech Lan- guage Translation (MSLT) English-French task. Our main finding is that translating conversation transcripts turned out to not be as challenging as we expected: while translation quality is of course not perfect, a straightforward phrase- based system trained on movie subtitles yields high BLEU scores (high 40s on the development set) and manual analy- sis of 100 examples showed that 61 of them were correctly translated, and errors were mostly local disfluencies in the remaining examples. 1. Introduction Our goal at the University of Maryland (UMD) for the 2016 IWSLT evaluation campaign [1] was to evaluate the strength and weaknesses of standard machine translation (MT) archi- tectures on the novel task of translating transcripts of Skype conversations from English into French. This is a challenging task for several reasons. First, only small amounts of conversation transcripts ( 45k words of En- glish) are available for system development. Second, we an- ticipate that Skype conversations diverge from the vast ma- jority of available training corpora along many dimensions, including differences in what is talked about (content, topics) and how it is talked about (style, register). Our primary task in building a MT system will therefore be to identify useful training examples from a large pool of diverse out-of-domain parallel corpora. Our MT systems uses a standard phrase-based architec- ture, outlined in Section 2. Our experiments focused on de- termining what kind of training data is most useful to trans- late this compared various of data selection techniques (Sec- tions 4) to make the most of the available data (Section 3), and finally conduct an error analysis (Section 5). Translating such conversation transcripts in these settings turned out to not be as challenging as we expected: while translation quality is of course not perfect, a straightforward phrase-based system trained on movie subtitles yields high BLEU scores (high 40s on the development set) and manual analysis of 100 examples showed that 61 of them were cor- rectly translated, and errors were mostly local disfluencies in the remaining examples. Small improvements in BLEU were obtained by using a simple heuristic to select more Skype- like examples. This approach was the only data selection approach that improved performance. Using language mod- eling and text formality criterion to select examples that we expected to be closer to Skype data did not improve BLEU. 2. Machine Translation Architecture 2.1. Core Configuration We use the Moses [2] statistical machine translation toolkit to build phrase-based MT systems. Training the MT systems was mostly done by following the standard Moses pipeline1 with default parameters: the maximum length of phrases was limited to 7 words, reordering was limited to 6 words skipped and the reordering model was specified as msd-bidirectional- fe. Word alignments were generated using fast …

Content truncated to 500 words