Context Feature Selection for Distributional Similarity

Context Feature Selection for Distributional Similarity Masato Hagiwara, Yasuhiro Ogawa, and Katsuhiko Toyama Graduate School of Information Science, Nagoya University Furo-cho, Chikusa-ku, Nagoya, JAPAN 464-8603 {hagiwara, yasuhiro, toyama}@kl.i.is.nagoya-u.ac.jp Abstract Distributional similarity is a widely used concept to capture the semantic relatedness of words in various NLP tasks. However, ac- curate similarity calculation requires a large number of contexts, which leads to imprac- tically high computational complexity. To alleviate the problem, we have investigated the effectiveness of automatic context selec- tion by applying feature selection methods explored mainly for text categorization. Our experiments on synonym acquisition have shown that while keeping or sometimes in- creasing the performance, we can drastically reduce the unique contexts up to 10% of the original size. We have also extended the measures so that they cover context cate- gories. The result shows a considerable cor- relation between the measures and the per- formance, enabling the automatic selection of effective context categories for distribu- tional similarity. 1 Introduction Semantic similarity of words is one of the most im- portant lexical knowledge for NLP tasks including word sense disambiguation and synonym acquisi- tion. To measure the semantic relatedness of words, a concept called distributional similarity has been widely used. Distributional similarity represents the relatedness of two words by the commonality of contexts the words share, based on the distributional hypothesis (Harris, 1985), which states that seman- tically similar words share similar contexts. A wide range of contextual information, such as surrounding words (Lowe and McDonald, 2000; Curran and Moens, 2002a), dependency or case structure (Hindle, 1990; Ruge, 1997; Lin, 1998), and dependency path (Lin and Pantel, 2001; Pado and Lapata, 2007), has been utilized for similar- ity calculation, and achieved considerable success. However, a major problem which arises when adopt- ing distributional similarity is that it easily yields a huge amount of unique contexts. This can lead to high dimensionality of context space, often up to the order of tens or hundreds of thousands, which makes the calculation computationally impractical. Be- cause not all of the contexts are useful, it is strongly required for the efficiency to eliminate the unwanted contexts to ease the expensive cost. To tackle this issue, Curran and Moens (2002b) suggest assigning an index vector of canonical at- tributes, i.e., a small number of representative el- ements extracted from the original vector, to each word. When the comparison is performed, canonical attributes of two target words are firstly consulted, and the original vectors are referred to only if the attributes have a match between them. However, it is not clear whether the condition for canonical at- tributes they adopted, i.e., that the attributes must be the most weighted subject, direct object, or indirect object, is optimal in terms of the performance. There are also some existing studies which paid attention to the comparison of context categories for synonym acquisition (Curran and Moens, 2002a; Hagiwara et al., 2006). However, they have con- ducted only a posteriori comparison based on perfor- mance evaluation, and we are …

Content truncated to 500 words