Assessing the Post-Editing Effort for Automatic and Semi-Automatic Translations of DVD Subtitles

Proceedings of Recent Advances in Natural Language Processing, pages 97–103, Hissar, Bulgaria, 12-14 September 2011. Assessing the Post-Editing Effort for Automatic and Semi-Automatic Translations of DVD Subtitles Sheila C. M. de Sousa, Wilker Aziz and Lucia Specia Research Group in Computational Linguistics University of Wolverhampton Stafford Street, Wolverhampton, WV1 1SB, UK {sheila.castilhomonteirodesousa, w.aziz, l.specia}@wlv.ac.uk Abstract With the increasing demand for fast and accurate audiovisual translation, subtitlers are starting to consider the use of trans- lation technologies to support their work. An important issue that arises from the use of such technologies is measuring how much effort needs to be put in by the subti- tler in post-editing (semi-)automatic trans- lations. In this paper we present an ob- jective way of measuring post-editing ef- fort in terms of time. In experiments with English-Portuguese subtitles, we measure the post-editing effort of texts translated using machine translation and transla- tion memory systems. We also contrast this effort against that of translating the texts without any tools. Results show that post-editing is on average 40% faster than translating subtitles from scratch. With our best system, more than 69% of the translations require little or no post- editing. 1 Introduction Automatic and semi-automatic translation have become a potential help in the subtitling indus- try due to the increasing demand for translations and the short time professionals have to deliver them. Many attempts have been made to trans- late subtitles automatically by using different Ma- chine Translation (MT) approaches such as Rule- Based (RBMT), Example-Based (EBMT), Statis- tical (SMT) and also Translation Memory (TM) systems. However, no previous work compares different approaches in terms of the effort that is required to post-edit the translations they produce. Additionally, the related work in the field does not provide an in-depth comparison between the effort needed to translate subtitles from scratch and the effort needed to post-edit a draft version produced using translation tools. The ability to objectively assess translation technology tools according to their post-editing effort is essential for a well informed decision among the large variety of tools available, as well as to ensure that such tools produce translations that require less effort to post-edit (PE) than the ef- fort that would be necessary to translate the same texts from scratch (HT). In this paper we compile a corpus of English – Brazilian Portuguese subtitles and we compare two different MT approaches as well as a TM sys- tem using this corpus. The translations obtained are post-edited and the original sentences are also translated from scratch, both using a tool specially designed to gather objective and subjective effort indicators: time spent on performing the task and qualitative assessments. Results show that trans- lators can greatly benefit from automatically ob- tained translations. The rest of this paper is organized as follows: Section 2 gives an overview of prior work; Sec- tion 3 describes our parallel corpus of subtitles; Section 4 describes how the experiments were per- formed; Section 5 presents the results; and Section 6 concludes the paper …

Content truncated to 500 words