Novel elicitation and annotation schemes for sentential and sub-sentential alignments of bitexts

Novel elicitation and annotation schemes for sentential and sub-sentential alignments of bitexts Yong Xu, Franc¸ois Yvon LIMSI, CNRS, Universit´e Paris-Saclay, F-91405 Orsay {yong, yvon}@limsi.fr Abstract Resources for evaluating sentence-level and word-level alignment algorithms are unsatisfactory. Regarding sentence alignments, the existing data is too scarce, especially when it comes to difficult bitexts, containing instances of non-literal translations. Regarding word-level alignments, most available hand-aligned data provide a complete annotation at the level of words that is difficult to exploit, for lack of a clear semantics for alignment links. In this study, we propose new methodologies for collecting human judgements on alignment links, which have been used to annotate 4 new data sets, at the sentence and at the word level. These will be released online, with the hope that they will prove useful to evaluate alignment software and quality estimation tools for automatic alignment. Keywords: Parallel corpora, Sentence Alignments, Word Alignments, Confidence Estimation 1. Introduction Bitext alignment consists of finding corresponding units in bitexts, where a bitext is defined as the association of two texts assumed to be mutual translations. Such a mapping can be established at various levels of granularity: between paragraphs, between sentences, between phrases, or be- tween words. Primarily because of the development of Sta- tistical Machine Translation (SMT) technologies (Brown et al., 1993), sentence-level and word-level alignments have been studied for a long time. In state-of-the-art phrase- based SMT, sentence alignment aims at providing parallel sentence pairs for word alignment which is an important component of the complete pipeline (Koehn et al., 2003). Their uses extend to many other natural language process- ing (NLP) applications. For instance, sentence alignment has been applied in translator training (Simard et al., 1993), translation checking (Macklovitch, 1994), language learn- ing (Nerbonne, 2000; Kraif and Tutin, 2011), and bilin- gual reading (Pillias and Cubaud, 2015). Word align- ment is employed in bilingual lexica extraction (Smadja et al., 1996), word sense disambiguation (Diab and Resnik, 2002), etc. Thanks to a sustained research effort, many alignment methods have been proposed. Two recent re- views of bitext alignment are in (Wu, 2010; Tiedemann, 2011). Manually annotated reference alignment data sets are valu- able resources for the development of alignment tech- niques. On the one hand, they can be used as the super- vision examples for the methods (M´ujdricza-Maydt et al., 2013; Blunsom and Cohn, 2006); on the other hand, they provide ways to directly evaluate automatic alignment qual- ity, and warrant the investigation of error patterns. How- ever, constructing manually annotated alignment data sets can be challenging. For some tasks, this can be due to a lack of a clear annotation scheme. For others, annotation schemes can vary a lot, depending on the targeted applica- tions, language pairs, etc. In this paper, we describe our contribution to manual sentence-level alignment annotations in Section 2., fol- lowed by word-level alignment annotations in Section 3.. For sentence alignment, the research community has reached a consensus on the annotation scheme (Tiedemann, 2011). But the resource is quite scarce for …

Content truncated to 500 words