Composing Word Vectors for Japanese Compound Words Using Bilingual Word Embeddings

Composing Word Vectors for Japanese Compound Words Using Bilingual Word Embeddings Teruo Hirabayashi Kanako Komiya Masayuki Asahara Hiroyuki Shinnou Ibaraki University 4-12-1, Nakanarusawa, Hitachi, Ibaraki, 316-8511, Japan {20nd303t, kanako.komiya.nlp, hiroyuki.shinnou.0828}@vc.ibaraki.ac.jp National Institute for Japanese Language and Linguistics 10-2 Midoricho, Tachikawa, Tokyo, Japan masayu-a@ninjal.ac.jp Abstract This study conducted an experiment to com- pare the word embeddings of a compound word and a word in Japanese on the same vec- tor space using bilingual word embeddings. Because Japanese does not have word delim- iters between words; thus various word defi- nitions exist according to dictionaries and cor- pora. We divided one corpus into words on the basis of two definitions, namely, shorter and ordinary words and longer compound words, and regarded two word-sequences as a parallel corpus of different languages. We then gen- erated word embeddings from the corpora of these languages and mapped the vectors into the common space using monolingual map- ping methods, a linear transformation matrix, and VecMap. We evaluated our methods by synonym ranking using a thesaurus. Further- more, we conducted experiments of two com- parative methods: (1) a method where the compound words were divided into words and the word embeddings were averaged and (2) a method where the word embeddings of the latter words are regarded as those of the com- pound words. The VecMap results with the supervised option outperformed that with the identical option, linear transformation matrix, and the latter word method, but could not beat the average method. 1 Introduction Japanese words have many definitions because Japanese does not have word delimiters between words, and word boundaries are unspecific. There- fore, the Japanese dictionary defines words individ- ually. Japanese has different word definitions ac- cording to each corpus and dictionary. The long unit for compound words and the short unit for words in UniDic1 (Maekawa et al., 2010) developed by the National Institute for Japanese Language and Lin- guistics (NINJAL) are some of them. For example, “いちご狩り, ichigo-gari, strawberry picking” is de- fined as one word (short unit), whereas “ぶどう狩 り, budou-gari, grape picking” is defined as a com- pound word (long unit) with two words (short unit)2 in UniDic. Due to the limit of the dictionary’s cover- age, a morphological analyzer using UniDic treats “ いちご狩り, ichigo-gari, strawberry picking” as one word and “ぶどう狩り, budou-gari, grape picking” as two words, making it impossible to directly com- pare the word meanings of these two words via word embeddings. Therefore, to address the word unit discrepancy issue, this study proposes the usage of bilingual word embeddings (BWEs), which is usually used for mapping the word embeddings of two different languages into the same vector space, to map the word embeddings of long and short units into a com- mon vector space. Using the BWE makes it easy to compare the word embeddings of “いちご狩り, ichigo-gari, strawberry picking” and “ぶどう狩り, budou-gari, grape picking” because both are on the same vector space. This situation is more convenient for many application systems like an information re- trieval …

Content truncated to 500 words