Construction of a Japanese Word Similarity Dataset

Construction of a Japanese Word Similarity Dataset Yuya Sakaizawa and Mamoru Komachi Tokyo Metropolitan University 6-6 Asahigaoka Hino city, Tokyo 191-0065, Japan {sakaizawa-yuya@ed.,komachi@}tmu.ac.jp Abstract An evaluation of distributed word representation is generally conducted using a word similarity task and/or a word analogy task. There are many datasets readily available for these tasks in English. However, evaluating distributed representation in languages that do not have such resources (e.g., Japanese) is difficult. Therefore, as a first step toward evaluating distributed representations in Japanese, we constructed a Japanese word similarity dataset. To the best of our knowledge, our dataset is the first resource that can be used to evaluate distributed representations in Japanese. Moreover, our dataset contains various parts of speech and includes rare words in addition to common words. Keywords: word embeddings, distributed representation, word similarity 1. Introduction Traditionally, a word is represented as a sparse vector in- dicating the word itself (one-hot vector) or the context of the word (distributional vector). However, both the one-hot notation and distributional notation suffer from data sparse- ness since dimensions of the word vector do not interact with each other. Distributed word representation addresses the data sparseness problem by constructing a dense vec- tor of a fixed length, wherein contexts are shared (or dis- tributed) across dimensions. Distributed word representa- tion is known to improve the performance of many NLP applications such as machine translation (Chen and Guo, 2015) and sentiment analysis (Tai et al., 2015) to name a few. The task to learn a distributed representation is called representation learning. However, evaluating the quality of learned distributed word representation itself is not straightforward. In language modeling, perplexity or cross-entropy is widely accepted as a de facto standard for intrinsic evaluation. In con- trast, distributed word representations include the additive (or compositional) property of the vectors, which cannot be assessed by perplexity. Moreover, perplexity makes little use of infrequent words; thus, it is not appropriate for eval- uating distributed presentations that try to represent them. Therefore, a word similarity task and/or a word analogy task are generally used to evaluate distributed word repre- sentations in the NLP literature. The former judges whether distributed word representations improve modeling con- texts, and the latter estimates how well the learned repre- sentations achieve the additive property. However, such re- sources other than for English (e.g., Japanese) seldom exist. In addition, most of these datasets comprise high-frequency nouns so that they tend not to include other parts of speech. Hence, previous data fail to evaluate word representations of other parts of speech, including content words such as verbs and adjectives. To address the problem of the lack of a dataset for evaluat- ing Japanese distributed word representations, we propose to build a Japanese dataset for the word similarity task. Currently at JustSystems Corporation. The main contributions of our work are as follows: • To the best of our knowledge, it is the first work that constructs a Japanese word similarity dataset. • The dataset contains various parts of speech …

Content truncated to 500 words