Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2678–2688, Osaka, Japan, December 11-17 2016. Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification Yuezhang Li, Ronghuo Zheng, Tian Tian, Zhiting Hu, Rahul Iyer, Katia Sycara Language Technology Institute, School of Computer Science Carnegie Mellon University yuezhanl@andrew.cmu.edu Abstract Existing work learning distributed representations of knowledge base entities has largely failed to incorporate rich categorical structure, and is unable to induce category representations. We propose a new framework that embeds entities and categories jointly into a semantic space, by integrating structured knowledge and taxonomy hierarchy from large knowledge bases. Our framework enables to compute meaningful semantic relatedness between entities and categories in a principled way, and can handle both single-word and multiple-word concepts. Our method shows significant improvement on the tasks of concept categorization and dataless hierarchical classification. 1 Introduction Hierarchies, most commonly represented as tree or directed acyclic graph (DAG) structures, provide a natural way to categorize and locate knowledge in large knowledge bases (KBs) . For example, Word- Net (Fellbaum, 1998), Freebase (Bollacker et al., 2008), and Wikipedia 1 use hierarchical taxonomy to organize entities into category hierarchies. These hierarchical categories could benefit applications such as object and concept categorization (Verma et al., 2012; Rothenh¨ausler and Sch¨utze, 2009), document classification (Gopal and Yang, 2013; Hu et al., 2016), and link prediction on knowledge graphs (Lin et al., 2015; Zhang et al., 2016). In all of these applications, it is essential to have a good representation of categories and entities as well as a good semantic relatedness measure. In this paper, we propose two models to learn distributed representations of both entities and categories from large-scale knowledge bases (KBs). The Category Embedding (CE) model extends the entity embedding method of (Hu et al., 2015) and induces category embeddings in addition to entity vectors. The Hierarchical Category Embedding (HCE) model further enhances the CE model by integrating category hierarchical structure. The resulting entity and category vectors effectively capture meaningful semantic relatedness between entities and categories. We train the category and entity vectors on Wikipedia, and evaluate our methods from two applica- tions: concept categorization (Baroni and Lenci, 2010) and dataless hierarchical classification (Song and Roth, 2014). Our method achieves state-of-the-art performance on both tasks. Figure 1 shows an overview of the research elements in this paper. In summary, we make the following contributions: • We incorporate category information into entity embeddings with the proposed CE model to get entity and category embeddings simultaneously. • We leverage category hierarchies and develop the HCE model to enhance the embedding quality. • We propose a new concept categorization method based on nearest neighbor classification. Our method avoids the granularity disparity issue of categories which has plagued traditional clustering- based methods. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ This work has been partially funded by ARO award #W911NF1210416 1https://www.wikipedia.org 2678 Figure 1: The organization of research elements …

Content truncated to 500 words