Statistical Analysis of Alignment Characteristics for Phrase-based Machine Translation

Statistical Analysis of Alignment Characteristics for Phrase-based Machine Translation Patrik Lambert and Simon Petitrenaud LIUM, University of Le Mans Avenue La¨ennec 72085 Le Mans Cedex 9, France patrik.lambert@lium.univ-lemans.fr simon.petit-renaud@lium.univ-lemans.fr Yanjun Ma and Andy Way CNGL, Dublin City University Glasnevin Dublin 9, Ireland yma@computing.dcu.ie away@computing.dcu.ie Abstract In most statistical machine translation (SMT) systems, bilingual segments are ex- tracted via word alignment. However, there lacks systematic study as to what alignment characteristics can benefit MT under specific experimental settings such as the language pair or the corpus size. In this paper we produce a set of alignments by directly tuning the alignment model ac- cording to alignment F-score and BLEU score in order to investigate the alignment characteristics that are helpful in trans- lation. We report results for a phrase- based SMT system on Chinese-to-English IWSLT data, and Spanish-to-English Eu- ropean Parliament data. With a statistical analysis into alignment characteristics that are correlated with BLEU score, we give alignment hints to improve BLEU score using a phrase-based SMT system and dif- ferent types of corpus. 1 Introduction Most statistical machine translation (SMT) sys- tems (e.g. phrase-based, n-gram-based) build their translation models from word alignments trained in a previous stage. Many papers have shown that intrinsic alignment quality is poorly correlated with MT quality (for example (Vilar et al., 2006)). Accordingly, some research has attempted to tune the alignment directly according to specific MT evaluation metrics (Lambert et al., 2007). In this paper we instead try to discover which alignment characteristics improve or worsen translation qual- ity by analysing the word alignment produced by c⃝ 2010 European Association for Machine Translation. the alignment model with different tuning crite- ria. The findings can potentially benefit our under- standing of existing SMT systems as well as de- signing novel word alignment models. A considerable amount of research effort has been devoted to the investigation of alignment characteristics that benefit MT. These characteris- tics include alignment precision and recall (Ayan and Dorr, 2006; Chen and Federico, 2006; Mari˜no et al., 2006; Fraser and Marcu, 2007), long- distance links (Vilar et al., 2006), unlinked words (Guzman et al., 2009; Lambert et al., 2009), etc. In most of the related papers some alignment charac- teristics are usually considered, and the impact on MT of alignments with different values for these characteristics is evaluated. In this work, we start from an initial alignment and tune it directly according to an intrinsic align- ment quality metric (F-score, see Section 3.4) and according to an extrinsic translation quality metric (BLEU score (Papineni et al., 2002)). In this way, we can investigate for any alignment characteristic how it is affected by the change of tuning crite- rion. If there exist alignment characteristics which are helpful in translation, they should not depend on the specific aligner used. However, they could depend on parameters such as the type of MT sys- tem, the language pair, or the corpus size or type. In this way we can study more systematically how the considered characteristics depend …

Content truncated to 500 words