SubCo: A Learner Translation Corpus of Human and Machine Subtitles

SubCo: A Learner Translation Corpus of Human and Machine Subtitles Jos´e Manuel Mart´ınez Mart´ınez, Mihaela Vela Universit¨at des Saarlandes Campus, 66123 Saarbr¨ucken, Germany {j.martinez,m.vela}@mx.uni-saarland.de Abstract In this paper, we present a freely available corpus of human and automatic translations of subtitles. The corpus comprises the original English subtitles (SRC), both human (HT) and machine translations (MT) into German, as well as post-editions (PE) of the MT output. HT and MT are annotated with errors. Moreover, human evaluation is included in HT, MT, and PE. Such a corpus is a valuable resource for both human and machine translation communities, enabling the direct comparison – in terms of errors and evaluation – between human and machine translations and post-edited machine translations. Keywords: subtitling; quality assessment; MT evaluation 1. Introduction This paper describes a freely available corpus1 consisting of original English subtitles (SRC) translated into German. The translations are produced by human translators and by SUMAT (Volk, 2008; M¨uller and Volk, 2013; Etchegoyhen et al., 2014), a MT system developed and trained specifi- cally for the translation of subtitles. Both human (HT) and machine translations (MT) are annotated with translation errors (HT ERROR, MT ERROR) as well as evaluation scores at subtitle level (HT EVAL, MT EVAL). Addition- ally, machine translations are completed with post-editions (PE) which are also evaluated (PE EVAL). The corpus was compiled as part of a course on subtitling targeted at students enrolled in the BA Translation Studies programme at Saarland University. The students carried out the human translations and the error and evaluation anno- tation. Though novice translators, this kind of evaluation is already an improvement, since human evaluation of MT datasets is often carried out by lay translators (computa- tional linguists or computer scientists). To the best of our knowledge, this is the first attempt to apply human error annotation and evaluation to subtitles produced by humans and a MT system, taking into account the distinctive features of subtitling such as temporal and spatial constrains. Such a resource can be useful for both human and machine translation communities. For (human) translation scholars a corpus of translated subtitles can be very helpful from a pedagogical point of view. In the field of MT, the possibility of comparing between human and machine translations, es- pecially using manual metrics like error analysis and eval- uation scores, can enhance MT development, in particular, applied to the specific task of subtitling. 2. Subtitling and Quality Assessment in Human and Machine Translation Subtitling implies taking into consideration that translated text has to be displayed synchronously with the image. This means that certain constrains on space (number of lines on 1http://hdl.handle.net/11858/ 00-246C-0000-0023-8D18-7 screen, number of characters per line) and time (synchrony, time displayed on screen) have to be respected when per- forming subtitling. According to D´ıaz Cintas and Remael (2007) subtitling is defined as a translation practice that consists of presenting a written text, generally on the lower part of the screen, that endeavours to recount the original di- alogue of the speakers as …

Content truncated to 500 words