Building an Arabic Machine Translation Post-Edited Corpus: Guidelines and Annotation

Building an Arabic Machine Translation Post-Edited Corpus: Guidelines and Annotation Wajdi Zaghouani, Nizar Habash†, Ossama Obeid, Behrang Mohit‡, Houda Bouamor and Kemal Oflazer Carnegie Mellon University in Qatar, †New York University Abu Dhabi, ‡Ask.com {wajdiz,oobeid,hbouamor}@cmu.edu, ko@cs.cmu.edu †nizar.habash@nyu.edu, ‡behrangm@ischool.berkeley.edu Abstract We present our guidelines and annotation procedure to create a human corrected machine translated post-edited corpus for the Modern Standard Arabic. Our overarching goal is to use the annotated corpus to develop automatic machine translation post-editing systems for Arabic that can be used to help accelerate the human revision process of translated texts. The creation of any manually annotated corpus usually presents many challenges. In order to address these challenges, we created comprehensive and simplified annotation guidelines which were used by a team of five annotators and one lead annotator. In order to ensure a high annotation agreement between the annotators, multiple training sessions were held and regular inter-annotator agreement measures were performed to check the annota- tion quality. The created corpus of manual post-edited translations of English to Arabic articles is the largest to date for this language pair. Keywords: Post-Editing, Guidelines, Annotation 1. Introduction In recent years, machine translation (MT) became widely used by translation companies to reduce their costs and im- prove their speed. Therefore, the demand for quick and accurate machine translations is growing. Machine trans- lation (MT) systems often produce incorrect output with many grammatical and lexical choice errors. Correcting machine-produced translation errors, or MT Post-Editing (PE) can be done automatically or manually. Successful au- tomatic post-editing approaches using manually corrected MT output were used by Elming (2006) and Simard et al. (2007). The availability of annotated resources is required for such approaches. When it comes to the Arabic lan- guage, to the best of our knowledge, there is no manually post-edited MT corpora available to build such systems. Therefore, there is a clear need to build such valuable re- sources for the Arabic language. In this paper, we present our guidelines and annotation pro- cedure to create a human corrected MT corpus for the Mod- ern Standard Arabic (MSA). The creation of any manually annotated corpus usually presents many challenges. In or- der to address these challenges, we created comprehensive and simplified annotation guidelines which were used by a team of five annotators and one lead annotator. In order to ensure a high annotation agreement between the annotators, multiple training sessions were held and regular inter anno- tator agreement (IAA) measures were performed to check the annotation quality. To the best of our knowledge, this is the first published machine translation manual post-editing annotation effort for Arabic of this scale. In the next sections, we review related work (Section 2), describe our corpus and the development of the guidelines (Sections 3-4), and present our annotation procedure (Sec- tion 5), than we present the annotation evaluation in Section 6, finally we conclude our work in Section 7. 2. Related Work Large scale manually corrected MT corpora are not yet widely available due to the high cost …

Content truncated to 500 words