Term-list Translation using Mono-lingual Word Co-occurrence Vectors

Term-list Translation using Mono-lingual Word Co-occurrence Vectors* Genichiro Kikui NTT Information and Communication Systems Labs. 1-1 Hikarinooka, Yokosuka-Shi, Kanagawa, Japan e-mail: kikui~is!.ntt.co.jp Abstract A term-list is a list of content words that charac- terize a consistent text or a concept. This paper presents a new method for translating a term-list by using a corpus in the target language. The method first retrieves alternative translations for each input word from a bilingual dictionary. It then determines the most 'coherent' combination of alternative trans- lations, where the coherence of a set of words is defined as the proximity among multi-dimensional vectors produced from the words on the basis of co-occurrence statistics. The method was applied to term-lists extracted from newspaper articles and achieved 81% translation accuracy for ambiguous words (i.e., words with multiple translations). 1 Introduction A list of content words, called a term-list, is widely used as a compact representation of documents in in- formation retrieval and other document processing. Automatic translation of term-lists enables this pro- cessing to be cross-linguistic. This paper presents a new method for translating term-lists by using co- occurrence statistics in the target language. Although there is little study on automatic trans- lation of term-lists, related studies are found in the area of target word selection (for content words) in conventional full-text machine translation (MT). Approaches for target word selection can be clas- sified into two types. The first type, which has been adopted in many commercial MT systems, is based on hand assembled disambiguation rules, and/or dic- tionaries. The problem with this approach is that creating these rules requires much cost and that they are usually domain-dependent 1 The second type, called the statistics-based ap- proach, learns disambiguation knowledge from large corpora. Brown et al. presented an algorithm that * This research was done when the author was at Center for the Study of Language and Information(CSLI), Stanford University. 1In fact, this is partly shown by the fact that many MT systems have substitutable domain-dependent (or "user" ) dic- tionaries . relies on translation probabilities estimated from large bilingual corpora (Brown et al., 1990)(Brown et al., 1991). Dagan and Itai (1994) and Tanaka and Iwasaki (1996) proposed algorithms for selecting tar- get words by using word co-occurrence statistics in the target language corpora. The latter algorithms using mono-lingual corpora are particularly impor- tant because, at present, we cannot always get a sufficient amount of bilingual or parallel corpora. Our method is closely related to (Tanaka and Iwasaki, 1996) from the viewpoint that they both rely on mono-lingual corpora only and do not re- quire any syntactic analysis. The difference is that our method uses "coherence scores", which can cap- ture associative relations between two words which do not co-occur in the training corpus. This paper is organized as follows, Section 2 de- scribes the overall translation process. Section 3 presents a disambiguation algorithm, which is the core part of our translation Inethod. Section 4 and 5 give experimental results and discussion. 2 Term-list Translation Our …

Content truncated to 500 words