Measuring the Similarity between Compound Nouns in Different Languages Using Non-Parallel Corpora

Measuring the Similarity between Compound Nouns in Different Languages Using Non-Parallel Corpora Takaaki TANAKA NTT Communication Science Laboratories 2-4 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, 619-0237, JAPAN takaaki@cslab.kecl.ntt.co.jp Abstract This paper presents a method that measures the similarity between compound nouns in different languages to locate translation equivalents from corpora. The method uses information from un- related corpora in different languages that do not have to be parallel. This means that many corpora can be used. The method compares the contexts of target compound nouns and trans- lation candidates in the word or semantic at- tribute level. In this paper, we show how this measuring method can be applied to select the best English translation candidate for Japanese compound nouns in more than 70% of the cases. 1 Introduction Many electronic documents in various languages are distributed via the Internet, CD-ROM, and other computer media. Cross-lingual natural language processing such as machine translation (MT) and cross-lingual information retrieval (CLIR) is becoming more important. When we read or write documents in a foreign language, we need more knowledge than what is provided in an ordinary dictionary, such as terminology, words relevant to current affairs, etc. Such expressions can be made up of multi- ple words, and there are almost infinite possible variations. Therefore, so it is quite difficult to add them and their translations to a dictionary. Many approaches have tried to acquire trans- lation equivalents automatically from parallel corpora (Dagan and Itai, 1994; Fung, 1995). In parallel corpora, effective features that have ob- vious correlations between these corpora can be used – e.g., similarity of position and frequency of words. However, we cannot always get enough par- allel corpora to extract the desired information. We propose a method of measuring the similar- ity to acquire compound noun translations by corpus information, which is not restricted to parallel corpora. Co-occurrence information is obtained as context where target nouns appear from the corpora. In a specific domain (e.g., fi- nancial news), a target word and its translation are often used in a similar context. For exam- ple, in a financial newspaper, price competition may appear with products (electric appliances, clothes, and foods), stores and companies more often than with nations and public facilities. 2 Extraction of Translations from Non-Parallel Corpora In parallel corpora, positions and frequencies of translation equivalents are correlated; there- fore, when we try to find translation equivalents from parallel corpora, this information provides valuable clues. On the other hand, in non- parallel corpora, positions and frequencies of words cannot be directly compared. Fung as- sumed that co-occurring words of translation equivalents are similar, and compared distri- butions of the co-occurring words to acquire Chinese-English translations from comparable corpora (Fung, 1997). This method generates co-occurring words vectors for target words, and judges the pair of words whose similarity is high to be translation equivalents. Rapp made German and English association word vectors and calculated the similarity of these vectors to find translations (Rapp, 1999). K.Tanaka and Iwasaki (1996) also assumed the resem- …

Content truncated to 500 words