Word Alignment of English-Chinese Bilingual Corpus Based on Chucks

Word Alignment of English-Chinese Bilingual Corpus Based on Chunks Sun Le, Jin Youbing, Du Lin, Sun Yufang Chinese Information Processing Center Institute of Software Chinese Academy of Sciences Beijing 100080 P. R. China lesun, ybjin, yfsun, ldu@sonata.iscas.ac.cn Abstract In this paper, a method for the word alignment of English-Chinese corpus based on chunks is proposed. The chunks of English sentences are identified firstly. Then the chunk boundaries of Chinese sentences are predicted by the translations of English chunks and heuristic information. The ambiguities of Chinese chunk boundaries are resolved by the coterminous words in English chunks. With the chunk aligned bilingual corpus, a translation relation probability is proposed to align words. Finally, we evaluate our system by real corpus and present the experiment results. Key Words: Word Alignment, Chunk Alignment, Bilingual Corpus, Lexicon Extraction 1 Introduction With the easier access to bilingual corpora, there is a tendency in NLP community to process and refine the bilingual corpora, which can serve as the knowledge base in support of many NLP applications, such as automatic or human-aid translation, multilingual terminology and lexicography, multilingual information retrieval system, etc. Different NLP applications need different bilingual corpora, which are aligned at different level. They can be divided by the nature of the segment to section level, paragraph level, sentence level, phrase level, word level, byte level, etc. As for our applications, we choose the chunk level to do alignment based on following considerations. Firstly, our applications, which include an example-based machine translation system, a computer aid translation system and a multilingual information retrieval system, need the alignment below the sentence level, on which we can acquire bilingual word and phrase dictionaries and. other useful translation information. Secondly, the word level alignment between English and Chinese language is difficult to deal with. There are no cognate words. The change in Chinese word order and word POS always produce many null and mistake correspondences. Next, we observe the phenomenon that when we translate the English sentence to Chinese sentence, all the words in one English chunk tend to be translated as one block of Chinese words which are coterminous. The word orders within these blocks tend to keep with the English chunk, also. So there are stronger boundaries between chunks than between words when we translate texts. Finally, as we all known, chunk has been assigned syntactic structure (Steven Abney, 1991), which comprises a connected sub-graph of the sentence's parse tree. So it's possible to align sentence structure and obtain translation grammars based on chunks by parsing. Many researchers have studied the text alignment problem and a number of quite encouraging results have been reported to different level alignments. With sentence-aligned corpus ready in hand, we focus our attention on the intra-sentence alignment between the sentence pairs. In this paper, a method for the word alignment of English-Chinese corpus based on chunks is proposed. The chunks of English sentences are identified firstly. Then the chunk boundaries of Chinese sentences are predicted by the bilingual lexicon and synonymy …

Content truncated to 500 words