A Part-of-Speech-Based Alignment Algorithm

A Part-of-Speech-Based Alignment Algorithm Kuang-hua Chen and tlsin-I Isi Chen Department of Computer Science and Information Engineering National Taiwan University Taipei, Taiwan, R.O.C. e-mail: hh chen@csie.ntu.edu.tw Abstract To align bilingual texts becomes a crucial issue recently. Rather than using length-based or translation-based criterion, a part-of-speech-based criterion is proposed. We postulate that source texts and target texts should share the same concepts, ideas, entities, and events. Simulated annealing approach is used to implement this aligmnent algorithm. The preliminary experiments show good performance. Most importantly, the experimental objects are Chinese-English texts, which are selected from different language families. 1. Introduction Real texts provide the alive phenomena, usages, and tendency of langnage in a parlictflar space and time. This recommends us to do the researches on the corpora. Recently, many rese~{rchers timber claim that "two languages art more informative than one" (Dagan, 1991). They show that two languages coukl disambigna.te each other (Gale e¢ al., 1992); bilingual corpus could form a bilingual dictionary (Brown et al., 1988) and terminology correspondence bank (Eijk, 1993); a refined bilingual corpus could be formed the examples for machine translation systems (Sumita et al., 1990). To do such kinds of researches, the most impmlant task is to align the bilingual texts. Many length-based alignment algorithms have been proposed (Brown et al., 1991; Gale and Church, 1991a). The correct rates are good. However, the languages they processed belong to occidental family. When these algorithms are applied to other rtmning texts from different families, will the performance keep on tile same level? Other translation-based alignments (Kay, 199l; Chen, 1993) show the difficulty in determining the word correspondence and are very complex. In tiffs paper, we will introduce a part-of-speech (POS)-based alignment algorithm. Section 2 will touch on the level of alignment and define the sentence terminators. In Section 3, we will propose tile criterion of critical POSes and investigate the distribution of these POSes in the Chinese-English texts. Section ,l will describe a fifir and rigorous method for evaluating performance. Then, we apply simulated annealing technique to conducting experiments and show tile experimental results in Section 5. Section 6 will give a brief conclusion. 2. Alignment Problem Alignment has three levels: 1) paragrapll; 2) sentence; and 3) word. Paragraph level is sometimes called discourse level. Many efforts are involved in senlence level and fewer researchers louch on the word level (Gale and Clmrch, 1991b). To do sentence alignment, we should first define what a sentence is. ILl English, tile sentence terminators are fifll stop, question mark, and exclamation mark. [[owever, tim usage of punctuation marks is unrestricted in Chinese and the types of punctuation marks are numerous (Yang, 1981). Nevertheless, in order to parallel the languages, we define that tile sentence markers are fldl slop, question mark, and exclamation mark over all languages. Therelbre, an alignment of two texts is to find a best sequence of sentcnce groups, which arc ended with one of tile sentence ternlillalors. Following Brown el al. (1991), we use tile term bead. A bead contains …

Content truncated to 500 words