Distributional Hypernym Generation by Jointly Learning Clusters and Projections

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1871–1879, Osaka, Japan, December 11-17 2016. Distributional Hypernym Generation by Jointly Learning Clusters and Projections Josuke Yamane† Tomoya Takatani‡ Hitoshi Yamada‡ Makoto Miwa† Yutaka Sasaki† † Toyota Technological Institute ‡ Toyota Motor Corporation † {sd16432,makoto-miwa,yutaka.sasaki}@toyota-ti.ac.jp ‡ {tomoya takatani,hitoshi yamada aa}@mail.toyota.co.jp Abstract We propose a novel word embedding-based hypernym generation model that jointly learns clus- ters of hyponym-hypernym relations, i.e., hypernymy, and projections from hyponym to hyper- nym embeddings. Most of the recent hypernym detection models focus on a hypernymy classifi- cation problem that determines whether a pair of words is in hypernymy or not. These models do not directly deal with a hypernym generation problem in that a model generates hypernyms for a given word. Differently from previous studies, our model jointly learns the clusters and pro- jections with adjusting the number of clusters so that the number of clusters can be determined depending on the learned projections and vice versa. Our model also boosts the performance by incorporating inner product-based similarity measures and negative examples, i.e., sampled non-hypernyms, into our objectives in learning. We evaluated our joint learning models on the task of Japanese and English hypernym generation and showed a significant improvement over an existing pipeline model. Our model also compared favorably to existing distributed hypernym detection models on the English hypernym classification task. 1 Introduction Hypernym-hyponym relations, a.k.a. hypernymy, are important information for several NLP tasks such as question answering and ontology construction. Some manually-constructed semantic resources like WordNet contain hypernymy; however, they have limited coverage. Plenty of studies have been con- ducted to automatically detect hypernymy, e.g., (Hearst, 1992; Roller et al., 2014; Fu et al., 2015). Hypernymy detection was traditionally often tackled with unsupervised methods using Hearst-style patterns (Hearst, 1992) or distributional inclusion hypothesis (Geffet and Dagan, 2005). These methods treat hypernymy pairs individually. Recent progress in the word representation allows to represent words in a shared low-dimensional vector space, and several models using the distributional word representation or word embeddings have been proposed for hypernymy detection (Roller et al., 2014; Weeds et al., 2014; Turney and Mohammad, 2015; Levy et al., 2015). Such models employ supervised learning. Most of the models focus on a hypernymy classification problem, i.e., whether a given word pair is in hypernymy or not, and they ignore a more practical hypernym generation problem to generate hypernyms for a given word. Few studies have examined hypernym generation using word embeddings (Fu et al., 2015; Tan et al., 2015). Fu et al. (2015) proposed a two-step, pipeline model that partitions hypernymy pairs into several clusters and learns a projection matrix between words in a pair for each cluster separately. The projection matrix projects the embedding vector of a hyponym close to that of its hypernym. The incorporation of clustering allows representing several types of hypernymy and shows a higher performance than other traditional models in constructing semantic hierarchies. The clusters, however, may not be appropriate for hypernym generation …

Content truncated to 500 words