Deep Joint Entity Disambiguation with Local Neural Attention

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2619–2629 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Deep Joint Entity Disambiguation with Local Neural Attention Octavian-Eugen Ganea and Thomas Hofmann Department of Computer Science ETH Zurich {octavian.ganea,thomas.hofmann}@inf.ethz.ch Abstract We propose a novel deep learning model for joint document-level entity disam- biguation, which leverages learned neural representations. Key components are en- tity embeddings, a neural attention mech- anism over local context windows, and a differentiable joint inference stage for dis- ambiguation. Our approach thereby com- bines benefits of deep learning with more traditional approaches such as graphical models and probabilistic mention-entity maps. Extensive experiments show that we are able to obtain competitive or state- of-the-art accuracy at moderate computa- tional costs. 1 Introduction Entity disambiguation (ED) is an important stage in text understanding which automatically re- solves references to entities in a given knowledge base (KB). This task is challenging due to the in- herent ambiguity between surface form mentions such as names and the entities they refer to. This many-to-many ambiguity can often be captured partially by name-entity co-occurrence counts ex- tracted from entity-linked corpora. ED research has largely focused on two types of contextual information for disambiguation: lo- cal information based on words that occur in a context window around an entity mention, and, global information, exploiting document-level co- herence of the referenced entities. Many state- of-the-art methods aim to combine the benefits of both, which is also the philosophy we follow in this paper. What is specific to our approach is that we use embeddings of entities as a common repre- sentation to assess local as well as global evidence. In recent years, many text and language under- standing tasks have been advanced by neural net- work architectures. However, despite recent work, competitive ED systems still largely employ man- ually designed features. Such features often rely on domain knowledge and may fail to capture all relevant statistical dependencies and interactions. The explicit goal of our work is to use deep learn- ing in order to learn basic features and their com- binations from scratch. To the best of our knowl- edge, our approach is the first to carry out this pro- gram with full rigor. 2 Contributions and Related Work There is a vast prior research on entity disam- biguation, highlighted by (Ji, 2016). We will focus here on a discussion of our main contributions in relation to prior work. Entity Embeddings. We have developed a sim- ple, yet effective method to embed entities and words in a common vector space. This fol- lows the popular line of work on word embed- dings, e.g. (Mikolov et al., 2013; Pennington et al., 2014), which was recently extended to entities and ED by (Yamada et al., 2016; Fang et al., 2016; Zwicklbauer et al., 2016; Huang et al., 2015). In contrast to the above methods that require data about entity-entity co-occurrences which of- ten suffers from sparsity, we rather bootstrap en- tity embeddings …

Content truncated to 500 words