Task-Oriented Learning of Word Embeddings for Semantic Relation Classification

Proceedings of the 19th Conference on Computational Language Learning, pages 268–278, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics Task-Oriented Learning of Word Embeddings for Semantic Relation Classification Kazuma Hashimoto†, Pontus Stenetorp‡, Makoto Miwa§, and Yoshimasa Tsuruoka† †The University of Tokyo, 3-7-1 Hongo, Bunkyo-ku, Tokyo, Japan {hassy,tsuruoka}@logos.t.u-tokyo.ac.jp ‡University College London, London, United Kingdom pontus@stenetorp.se §Toyota Technological Institute, 2-12-1 Hisakata, Tempaku-ku, Nagoya, Japan makoto-miwa@toyota-ti.ac.jp Abstract We present a novel learning method for word embeddings designed for relation classification. Our word embeddings are trained by predicting words between noun pairs using lexical relation-specific fea- tures on a large unlabeled corpus. This al- lows us to explicitly incorporate relation- specific information into the word embed- dings. The learned word embeddings are then used to construct feature vectors for a relation classification model. On a well- established semantic relation classification task, our method significantly outperforms a baseline based on a previously intro- duced word embedding method, and com- pares favorably to previous state-of-the-art models that use syntactic information or manually constructed external resources. 1 Introduction Automatic classification of semantic relations has a variety of applications, such as information ex- traction and the construction of semantic net- works (Girju et al., 2007; Hendrickx et al., 2010). A traditional approach to relation classification is to train classifiers using various kinds of features with class labels annotated by humans. Carefully crafted features derived from lexical, syntactic, and semantic resources play a significant role in achieving high accuracy for semantic relation clas- sification (Rink and Harabagiu, 2010). In recent years there has been an increasing in- terest in using word embeddings as an alternative to traditional hand-crafted features. Word embed- dings are represented as real-valued vectors and capture syntactic and semantic similarity between words. For example, word2vec1 (Mikolov et al., 2013b) is a well-established tool for learning word embeddings. Although word2vec has successfully been used to learn word embeddings, these kinds of word embeddings capture only co-occurrence relationships between words (Levy and Gold- berg, 2014). While simply adding word embed- dings trained using window-based contexts as ad- ditional features to existing systems has proven valuable (Turian et al., 2010), more recent studies have focused on how to tune and enhance word embeddings for specific tasks (Bansal et al., 2014; Boros et al., 2014; Chen et al., 2014; Guo et al., 2014; Nguyen and Grishman, 2014) and we con- tinue this line of research for the task of relation classification. In this work we present a learning method for word embeddings specifically designed to be use- ful for relation classification. The overview of our system and the embedding learning process are shown in Figure 1. First we train word em- beddings by predicting each of the words between noun pairs using lexical relation-specific features on a large unlabeled corpus. We then use the word embeddings to construct lexical feature vectors for relation classification. Lastly, the feature vectors are used to train a relation classification model. We evaluate our method on a well-established semantic relation classification task and …

Content truncated to 500 words