Design Challenges for Entity Linking

Design Challenges for Entity Linking Xiao Ling Sameer Singh University of Washington, Seattle WA {xiaoling,sameer,weld}@cs.washington.edu Daniel S. Weld Abstract Recent research on entity linking (EL) has in- troduced a plethora of promising techniques, ranging from deep neural networks to joint in- ference. But despite numerous papers there is surprisingly little understanding of the state of the art in EL. We attack this confusion by analyzing differences between several versions of the EL problem and presenting a simple yet effective, modular, unsupervised system, called VINCULUM, for entity linking. We con- duct an extensive evaluation on nine data sets, comparing VINCULUM with two state-of-the- art systems, and elucidate key aspects of the system that include mention extraction, candi- date generation, entity type prediction, entity coreference, and coherence. 1 Introduction Entity Linking (EL) is a central task in information extraction — given a textual passage, identify entity mentions (substrings corresponding to world entities) and link them to the corresponding entry in a given Knowledge Base (KB, e.g. Wikipedia or Freebase). For example, JetBlue begins direct service between Barnstable Airport and JFK International. Here, “JetBlue” should be linked to the en- tity KB:JetBlue, “Barnstable Airport” KB:Barnstable Municipal Airport, andto “JFK International” to KB:John F. Kennedy International Airport1. The links not only 1We use typewriter font, e.g., KB:Entity, to indicate an entity in a particular KB, and quotes, e.g., “Mention”, to denote textual mentions. provide semantic annotations to human readers but also a machine-consumable representation of the most basic semantic knowledge in the text. Many other NLP applications can benefit from such links, such as distantly-supervised relation extraction (Craven and Kumlien, 1999; Riedel et al., 2010; Hoffmann et al., 2011; Koch et al., 2014) that uses EL to create training data, and some coreference systems that use EL for disambiguation (Hajishirzi et al., 2013; Zheng et al., 2013; Durrett and Klein, 2014). Unfortunately, in spite of numerous papers on the topic and several published data sets, there is surprisingly little understanding about state-of-the-art performance. We argue that there are three reasons for this con- fusion. First, there is no standard definition of the problem. A few variants have been studied in the liter- ature, such as Wikification (Milne and Witten, 2008; Ratinov et al., 2011; Cheng and Roth, 2013) which aims at linking noun phrases to Wikipedia entities and Named Entity Linking (aka Named Entity Dis- ambiguation) (McNamee and Dang, 2009; Hoffart et al., 2011) which targets only named entities. Here we use the term Entity Linking as a unified name for both problems, and Named Entity Linking (NEL) for the subproblem of linking only named entities. But names are just one part of the problem. For many variants there are no annotation guidelines for scor- ing links. What types of entities are valid targets? When multiple entities are plausible for annotating a mention, which one should be chosen? Are nested mentions allowed? Without agreement on these is- sues, a fair comparison is elusive. Secondly, it is almost impossible to assess ap- proaches, because systems …

Content truncated to 500 words