Representation Learning of Entities and Documents from Knowledge Base Descriptions

Proceedings of the 27th International Conference on Computational Linguistics, pages 190–201 Santa Fe, New Mexico, USA, August 20-26, 2018. 190 Representation Learning of Entities and Documents from Knowledge Base Descriptions Ikuya Yamada1,3 Hiroyuki Shindo2,3 Yoshiyasu Takefuji4 ikuya@ousia.jp shindo@is.naist.jp takefuji@sfc.keio.ac.jp 1 Studio Ousia, 2 Nara Institute of Science and Technology, 3 RIKEN AIP, 4 Keio University Abstract In this paper, we describe TextEnt, a neural network model that learns distributed representations of entities and documents directly from a knowledge base (KB). Given a document in a KB consisting of words and entity annotations, we train our model to predict the entity that the document describes and map the document and its target entity close to each other in a continuous vector space. Our model is trained using a large number of documents extracted from Wikipedia. The performance of the proposed model is evaluated using two tasks, namely fine-grained entity typing and multiclass text classification. The results demonstrate that our model achieves state- of-the-art performance on both tasks. The code and the trained representations are made available online for further academic research. 1 Introduction The problem of learning distributed representations (or embeddings) from a knowledge base (KB) has recently attracted considerable attention. These representations enable us to use the large-scale, human- edited information of a KB in machine learning models, and can be applied in various natural language tasks such as entity linking (Hu et al., 2015; Yamada et al., 2016; Yamada et al., 2017), entity search (Hu et al., 2015), and link prediction (Bordes et al., 2013; Wang et al., 2014). In this paper, we describe TextEnt, a simple neural network model that learns distributed representa- tions of entities and documents from a KB. Specifically, given a document in a KB consisting of words and contextual entities (i.e., entities referred from entity annotations in the document), our model predicts the target entity explained by the document (see Figure 1), and maps the document and its target entity close to each other in a continuous vector space. Here, words, contextual entities, and target entities are mapped into continuous vectors that are updated throughout the training. In this study, we train the model using documents retrieved from Wikipedia. One key characteristic of our model is that it enables us to combine the semantic signals obtained from both words and entities in a straightforward manner. The main motivation for using entities in addition to words is to address the problems of ambiguity (i.e., the same words or phrases may have different meanings) and variety (i.e., the same meaning may be expressed using different words or phrases) in natural language. For example, the word Washington is ambiguous because it can refer to a US state, or the capital city of the US, or the first US president George Washington, and so on. Further, New York is sometimes referred to as NY or by its nickname, the Big Apple. Obviously, entities do not have these problems, because they are uniquely identified in the KB. To evaluate …

Content truncated to 500 words