Entity Hierarchy Embedding

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1292–1300, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Entity Hierarchy Embedding Zhiting Hu, Poyao Huang, Yuntian Deng, Yingkai Gao, Eric P. Xing School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA {zhitingh,poyaoh,yuntiand,yingkaig,epxing}@cs.cmu.edu Abstract Existing distributed representations are limited in utilizing structured knowledge to improve semantic relatedness modeling. We propose a principled framework of em- bedding entities that integrates hierarchi- cal information from large-scale knowl- edge bases. The novel embedding model associates each category node of the hi- erarchy with a distance metric. To cap- ture structured semantics, the entity sim- ilarity of context prediction are measured under the aggregated metrics of relevant categories along all inter-entity paths. We show that both the entity vectors and cat- egory distance metrics encode meaningful semantics. Experiments in entity linking and entity search show superiority of the proposed method. 1 Introduction There has been a growing interest in distributed representation that learns compact vectors (a.k.a embedding) for words (Mikolov et al., 2013a), phrases (Passos et al., 2014), and concepts (Hill and Korhonen, 2014), etc. The induced vectors are expected to capture semantic relatedness of the linguistic items, and are widely used in senti- ment analysis (Tang et al., 2014), machine transla- tion (Zhang et al., 2014), and information retrieval (Clinchant and Perronnin, 2013), to name a few. Despite the impressive success, existing work is still limited in utilizing structured knowledge to enhance the representation. For instance, word and phrase embeddings are largely induced from plain text. Though recent knowledge graph em- beddings (Lin et al., 2015; Wang et al., 2014) inte- grate the relational structure among entities, they primarily target at link prediction and lack an ex- plicit relatedness measure. In this paper, we propose to improve the dis- tributed representations of entities by integrating hierarchical information from large-scale knowl- edge bases (KBs). An entity hierarchy groups en- tities into categories which are further organized to form a taxonomy. It provides rich structured knowledge on entity relatedness (Resnik, 1995). Our work goes beyond the previous heuristic use of entity hierarchy which relies on hand-crafted features (Kaptein and Kamps, 2013; Ponzetto and Strube, 2007), and develops a principled optimization-based framework. We learn a dis- tance metric for each category node, and mea- sure entity-context similarity under the aggregat- ed metrics of all relevant categories. The met- ric aggregation encodes the hierarchical property that nearby entities tend to share common seman- tic features. We further provide a highly-efficient implementation in order to handle large complex hierarchies. We train a distributed representation for the w- hole entity hierarchy of Wikipedia. Both the entity vectors and the category distance metrics capture meaningful semantics. We deploy the embedding in both entity linking (Han and Sun, 2012) and entity search (Demartini et al., 2010) tasks. Hi- erarchy embedding significantly outperforms that without structural knowledge. Our methods also show superiority over existing competitors. …

Content truncated to 500 words