Integrating Cross-Lingually Relevant News Articles and Monolingual Web Documents in Bilingual Lexicon Acquisition

Integrating Cross-Lingually Relevant News Articles and Monolingual Web Documents in Bilingual Lexicon Acquisition Takehito Utsuro† and Kohei Hino‡ and Mitsuhiro Kida† Seiichi Nakagawa‡ and Satoshi Sato† †Graduate School of Informatics, Kyoto University, Sakyo-ku, Kyoto, 606-8501, Japan ‡Department of Information and Computer Sciences, Toyohashi University of Technology Tenpaku-cho, Toyohashi, 441–8580, Japan Abstract In the framework of bilingual lexicon acquisition from cross-lingually relevant news articles on the Web, it is relatively harder to reliably estimate bilin- gual term correspondences for low frequency terms. Considering such a situation, this paper proposes to complementarily use much larger monolingual Web documents collected by search engines, as a resource for reliably re-estimating bilingual term correspon- dences. We experimentally show that, using a suf- ficient number of monolingual Web documents, it is quite possible to have reliable estimate of bilin- gual term correspondences for those low frequency terms. 1 Introduction Translation knowledge acquisition from paral- lel/comparative corpora is one of the most im- portant research topics of corpus-based MT. This is because it is necessary for an MT sys- tem to (semi-)automatically increase its trans- lation knowledge in order for it to be used in the real world situation. One limitation of the corpus-based translation knowledge acquisi- tion approach is that the techniques of transla- tion knowledge acquisition heavily rely on avail- ability of parallel/comparative corpora. How- ever, the sizes as well as the domain of existing parallel/comparative corpora are limited, while it is very expensive to manually collect paral- lel/comparative corpora. Therefore, it is quite important to overcome this resource scarcity bottleneck in corpus-based translation knowl- edge acquisition research. In order to solve this problem, we proposed an approach of taking bilingual news articles on Web news sites as a source for translation knowledge acquisition (Utsuro et al., 2003). In the case of Web news sites in Japan, Japanese as well as English news articles are updated ev- eryday. Although most of those bilingual news articles are not parallel even if they are from the same site, certain portion of those bilingual news articles share their contents or at least re- port quite relevant topics. This characteristic is quite important for the purpose of transla- tion knowledge acquisition. Utsuro et al. (2003) showed that it is possible to acquire translation knowledge of domain specific named entities, event expressions, and collocational expressions from the collection of bilingual news articles on Web news sites. Based on the results of our previous study, this paper further examines the correlation of term frequency and the reliability of bilingual term correspondences estimated from bilingual news articles. We show that, for high frequency terms, it is relatively easier to reliably estimate bilingual term correspondences. However, for low frequency terms, it is relatively harder to re- liably estimate bilingual term correspondences. Low frequency problem of this type often hap- pens when a sufficient number of bilingual news articles are not available at hand. Considering such a situation, this paper then proposes to complementarily use much larger monolingual Web documents collected by search engines, …

Content truncated to 500 words