A Robust Cross-Style Bilingual Sentences Alignment Model

A Robust Cross-Style Bilingual Sentences Alignment Model Tz-Liang Kueng Keh-Yih Su Behavior Design Corporation 2F, No.5, Industry E. Rd. IV, Science-Based Industrial Park, Hsinchu, Taiwan 30077, R.O.C. {cavs, kysu}@bdc.com.tw Abstract Most current sentence alignment approaches adopt sentence length and cognate as the alignment features; and they are mostly trained and tested in the docu- ments with the same style. Since the length distribu- tion, alignment-type distribution (used by length-based approaches) and cognate frequency vary significantly across texts with different styles, the length-based ap- proaches fail to achieve similar performance when tested in corpora of different styles. The experiments show that the performance in F-measure could drop from 98.2% to 85.6% when a length-based approach is trained by a technical manual and then tested on a general magazine. Since a large percentage of content words in the source text would be translated into the corresponding trans- lation duals to preserve the meaning in the target text, transfer lexicons are usually regarded as more reliable cues for aligning sentences when the alignment task is performed by human. To enhance the robustness, a robust statistical model based on both transfer lexicons and sentence lengths are proposed in this paper. Af- ter integrating the transfer lexicons into the model, a 60% F-measure error reduction (from 14.4% to 5.8%) is observed. 1 Introduction Since the bilingual corpus is a valuable resource for training statistical language models [Dagon, 91; Su et al., 95; Su and Chang, 99] and sentence align- ment is the first step for most such tasks, many alignment approaches have been proposed in the literature [Brown, 91; Gale and Church, 93; Wu, 94; Vogel et al., 96; Och and Ney, 2000]. Most of those reported approaches use the sentence length as the main feature to perform the alignment task. For example, Brown et al. (91) used the feature of number-of-words for alignment, and [Gale and Church,93] claimed that better performance can be achieved (5.8% error rate for English-French cor- pus) if the number-of-characters is adopted instead. As cognates are reliable cues for language pairs de- rived from the same family, Church (93) also at- tacked this problem by considering cognates addi- tionally. Because most of those reported work are performed on those Indo-European language-pairs, for testing the performance on non-Indo-European languages, Wu (94) had tried both length and cog- nate features on the Hong Kong Hansard English- Chinese corpus, and 7.9% error rate has been re- ported. Besides, sentence alignment can also be indirectly achieved via more complicated word cor- responding models [Brown et al., 93; Vogel et al., 96; Och and Ney, 2000]. Since those word corre- sponding models, which also achieve similar per- formance, are more complicated and run relatively slow, they seems to be over-killed for the task of aligning sentences and will not be discussed in this paper. Although length-based approaches above men- tioned are simple and can achieve good perfor- mance, they are usually trained and tested in the text with the same style. Therefore, they are style-dependent …

Content truncated to 500 words