Efficient One-Pass End-to-End Entity Linking for Questions

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6433–6441, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6433 Efficient One-Pass End-to-End Entity Linking for Questions Belinda Z. Li♠∗ Sewon Min♦ Srinivasan Iyer† Yashar Mehdad† Wen-tau Yih† ♠MIT CSAIL ♦University of Washington †Facebook AI bzl@mit.edu sewon@cs.washington.edu {sviyer,mehdad,scottyih}@fb.com Abstract We present ELQ, a fast end-to-end entity link- ing model for questions, which uses a bien- coder to jointly perform mention detection and linking in one pass. Evaluated on WebQSP and GraphQuestions with extended annota- tions that cover multiple entities per question, ELQ outperforms the previous state of the art by a large margin of +12.7% and +19.6% F1, respectively. With a very fast inference time (1.57 examples/s on a single CPU), ELQ can be useful for downstream question answering systems. In a proof-of-concept experiment, we demonstrate that using ELQ significantly improves the downstream QA performance of GraphRetriever (Min et al., 2019).1 1 Introduction Entity linking (EL), the task of identifying enti- ties and mapping them to the correct entries in a database, is crucial for analyzing factoid ques- tions and for building robust question answering (QA) systems. For instance, the question “when did shaq come to the nba?” can be answered by ex- amining Shaquille O’Neal’s Wikipedia article (Min et al., 2019), or its properties in a knowledge graph (Yih et al., 2015; Yu et al., 2017). How- ever, real-world user questions are invariably noisy and ill-formed, lacking cues provided by casing and punctuation, which prove challenging to cur- rent end-to-end entity linking systems (Yang and Chang, 2015; Sorokin and Gurevych, 2018). While recent pre-trained models have proven highly ef- fective for entity linking (Logeswaran et al., 2019; Wu et al., 2020), they are only designed for entity disambiguation and require mention boundaries to be given in the input. Additionally, such systems ∗Work done while at Facebook AI. 1Code and data available at https://github.com/ facebookresearch/BLINK/tree/master/elq Figure 1: Overview of our end-to-end entity linking system. We separately encode the question and entity. We use the question representations to jointly detect mentions and score candidate entities through inner- product with the entity vector. have only been evaluated on long, well-formed doc- uments like news articles (Ji et al., 2010), but not on short, noisy text. Also, most prior works have focused mainly on improving model prediction ac- curacy, largely overlooking efficiency. In this work, we propose ELQ, a fast and accu- rate entity linking system that specifically targets questions. Following the Wikification setup (Rati- nov et al., 2011), ELQ aims to identify the mention boundaries of entities in a given question and their corresponding Wikipedia entity. We employ a bi- encoder based on BERT (Devlin et al., 2019) as shown in Figure 1. The entity encoder computes en- tity embeddings for all entities in Wikipedia, using their short descriptions. Then, the question encoder derives token-level embeddings for the input ques- tion. We detect mention boundaries using these embeddings, and disambiguate each entity mention based on an inner product between …

Content truncated to 500 words