Learning Term Embeddings for Taxonomic Relation Identification Using Dynamic Weighting Neural Network

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 403–413, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Learning Term Embeddings for Taxonomic Relation Identification Using Dynamic Weighting Neural Network Luu Anh Tuan Institute for Infocomm Research, Singapore at.luu@i2r.a-star.edu.sg Yi Tay Nanyang Technological University ytay2@e.ntu.edu.sg Siu Cheung Hui Nanyang Technological University asschui@ntu.edu.sg See Kiong Ng Institute for Infocomm Research, Singapore skng@i2r.a-star.edu.sg Abstract Taxonomic relation identification aims to rec- ognize the ‘is-a’ relation between two terms. Previous works on identifying taxonomic re- lations are mostly based on statistical and lin- guistic approaches, but the accuracy of these approaches is far from satisfactory. In this pa- per, we propose a novel supervised learning approach for identifying taxonomic relations using term embeddings. For this purpose, we first design a dynamic weighting neural net- work to learn term embeddings based on not only the hypernym and hyponym terms, but also the contextual information between them. We then apply such embeddings as features to identify taxonomic relations using a super- vised method. The experimental results show that our proposed approach significantly out- performs other state-of-the-art methods by 9% to 13% in terms of accuracy for both general and specific domain datasets. 1 Introduction Taxonomies which serve as the backbone of struc- tured knowledge are useful for many NLP applica- tions such as question answering (Harabagiu et al., 2003) and document clustering (Fodeh et al., 2011). However, the hand-crafted, well-structured tax- onomies including WordNet (Miller, 1995), Open- Cyc (Matuszek et al., 2006) and Freebase (Bol- lacker et al., 2008) that are publicly available may not be complete for new or specialized domains. It is also time-consuming and error prone to identify taxonomic relations manually. As such, methods for automatic identification of taxonomic relations is highly desirable. The previous methods for identifying taxonomic relations can be generally classified into two cate- gories: statistical and linguistic approaches. The sta- tistical approaches rely on the idea that frequently co-occurring terms are likely to have taxonomic re- lationships. While such approaches can result in taxonomies with relatively high coverage, they are usually heavily dependent on the choice of feature types, and suffer from low accuracy. The linguis- tic approaches which are based on lexical-syntactic patterns (e.g. ‘A such as B’) are simple and efficient. However, they usually suffer from low precision and coverage because the identified patterns are unable to cover the wide range of complex linguistic struc- tures, and the ambiguity of natural language com- pounded by data sparsity makes these approaches less robust. Word embedding (Bengio et al., 2001), also known as distributed word representation, which represents words with high-dimensional and real- valued vectors, has been shown to be effective in exploring both linguistic and semantic relations be- tween words. In recent years, word embedding has been used quite extensively in NLP research, rang- ing from syntactic parsing (Socher et al., 2013a), machine translation (Zou et al., 2013) to senti- ment analysis (Socher et al., 2013b). The cur- rent methods for learning …

Content truncated to 500 words