Integrated Use of Internal and External Evidence in the Alignment of Multi-Word Named Entities

Integrated Use of Internal and External Evidence in the Alignment of Multi-Word Named Entities Takeshi KUTSUMI SHARP Corporation 492 Minosho-cho, Yamatokoriyama-shi, Nara, 639-1186, JAPAN kutsumi.takeshi@sharp.co.jp Takehiko YOSHIMI Ryukoku University 1-5 Yokotani, Setaoe-cho, Otsu-shi, Shiga, 520-2925, JAPAN yoshimi@rins.ryukoku.ac.jp Katsunori KOTANI National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, JAPAN kat@khn.nict.go.jp Ichiko SATA SHARP Corporation 492 Minosho-cho, Yamatokoriyama-shi, Nara, 639-1186, JAPAN sata@isl.nara.sharp.co.jp Hitoshi ISAHARA National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, JAPAN isahara@nict.go.jp Abstract This paper proposes a method of extracting English multi-word named entities and their Japanese equivalents from a parallel corpus. The aim of our research is to extract multi-word named entities which are not listed in a dictionary of an English-to-Japanese MT system and appear infrequently in a parallel corpus. Our method makes its alignment on the basis of two kinds of external evidence provided by the context in which a bilingual pair appears, as well as two kinds of internal evidence within the pair. Each evidence is accompanied by a score, and the aggregate score is computed as a weighted sum of the scores. The appropriate weights are estimated with the logistic regression analysis. An experiment using a parallel corpus of Yomiuri Shimbun and The Daily Yomiuri satisfactorily found that 86.36% of the extracted bilingual pairs with the highest scores were judged to be correct. 1 Introduction Over the past few decades, English-to-Japanese bilingual dictionaries have been expanded both in quality and in quantity. The development has led to the practical use of MT systems which have the dictionaries of more than two millions of entries. But words which are not found even in such large-scale dictionaries can appear in real life texts. Since they are not found in the large dictionaries, they would appear infrequently in corpora. Many researches have pursued the extraction of bilingual pairs from a parallel corpus in which sentences are aligned (Eijk 1993; Kupiec 1993; Dekai & Xia 1994; Smadja & McKeown 1996; Ker & Chang 1997; Le et al. 1999; Huang et al. 2003), as well as from a comparable corpus in which sentences are not aligned (e.g. Tanaka 1999; Nakagawa 2001; Chiao 2002). Furthermore, some researchers have focused on the extraction of multi-word named entity translations (Al-Onaizan 2002; Moore 2003). The methods of extracting the bilingual pairs from a parallel corpus seem to have been established. However, in the case of aiming at extraction of the bilingual pair of a word with low frequency of occurrence and its counterpart, as is well known, statistical methods often encounter serious problems. Although literature on this task is available (Dagan & Church 1994; Tsuji 2001), it calls for further investigation to determine what kinds of linguistic information we should use and how we should combine them. PACLIC 18, December 8th-10th, 2004, Waseda University, Tokyo - 187 - We propose a method of extracting the bilingual pair of an English multi-word named entity (hereafter MWNE) and its Japanese counterpart. Here we focus our attention …

Content truncated to 500 words