Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 250–259, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation Ikuya Yamada1 4 Hiroyuki Shindo2 Hideaki Takeda3 Yoshiyasu Takefuji4 ikuya@ousia.jp shindo@is.naist.jp takeda@nii.ac.jp takefuji@sfc.keio.ac.jp 1Studio Ousia, 4489-105-221 Endo, Fujisawa, Kanagawa, Japan 2Nara Institute of Science and Technology, 8916-5 Takayama, Ikoma, Nara, Japan 3National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda, Tokyo, Japan 4Keio University, 5322 Endo, Fujisawa, Kanagawa, Japan Abstract Named Entity Disambiguation (NED) refers to the task of resolving multiple named entity mentions in a document to their correct references in a knowledge base (KB) (e.g., Wikipedia). In this paper, we propose a novel embedding method specifically designed for NED. The pro- posed method jointly maps words and enti- ties into the same continuous vector space. We extend the skip-gram model by us- ing two models. The KB graph model learns the relatedness of entities using the link structure of the KB, whereas the an- chor context model aims to align vectors such that similar words and entities occur close to one another in the vector space by leveraging KB anchors and their con- text words. By combining contexts based on the proposed embedding with standard NED features, we achieved state-of-the- art accuracy of 93.1% on the standard CoNLL dataset and 85.2% on the TAC 2010 dataset. 1 Introduction Named Entity Disambiguation (NED) is the task of resolving ambiguous mentions of entities to their referent entities in a knowledge base (KB) (e.g., Wikipedia). NED has lately been extensively studied (Cucerzan, 2007; Mihalcea and Csomai, 2007; Milne and Witten, 2008b; Ratinov et al., 2011) and used as a fundamental component in numerous tasks, such as information extraction, knowledge base population (McNamee and Dang, 2009; Ji et al., 2010), and semantic search (Blanco et al., 2015). We use Wikipedia as KB in this pa- per. The main difficulty in NED is ambiguity in the meaning of entity mentions. For example, the mention “Washington” in a document can refer to various entities, such as the state, or the capi- tal of the US, the actor Denzel Washington, the first US president George Washington, and so on. In order to resolve these ambiguous men- tions into references to the correct entities, early approaches focused on modeling textual context, such as the similarity between contextual words and encyclopedic descriptions of a candidate en- tity (Bunescu and Pasca, 2006; Mihalcea and Cso- mai, 2007). Most state-of-the-art methods use more sophisticated global approaches, where all mentions in a document are simultaneously disam- biguated based on global coherence among disam- biguation decisions. Word embedding methods are also becom- ing increasingly popular (Mikolov et al., 2013a; Mikolov et al., 2013b; Pennington et al., 2014). These involve learning continuous vector repre- sentations of words from large, unstructured text corpora. The vectors are designed to capture the semantic similarity of words when similar words are placed near one another in a relatively low- dimensional vector …

Content truncated to 500 words