Contextualized End-to-End Neural Entity Linking

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 637–642 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 637 Contextualized End-to-End Neural Entity Linking Haotian Chen BlackRock haotian.chen@blackrock.com Andrej Zukov-Gregoric BlackRock andrej.zukovgregoric@blackrock.com Xi (David) Li BlackRock david.li@blackrock.com Sahil Wadhwa University of Illinois at Urbana-Champaign∗ sahilw2@illinois.edu Abstract We propose an entity linking (EL) model that jointly learns mention detection (MD) and en- tity disambiguation (ED). Our model applies task-specific heads on top of shared BERT contextualized embeddings. We achieve state- of-the-art results across a standard EL dataset using our model; we also study our model’s performance under the setting when hand- crafted entity candidate sets are not available and find that the model performs well under such a setting also. 1 Introduction Entity linking (EL)1, in our context, refers to the joint task of recognizing named entity men- tions in text through mention detection (MD) and linking each mention to a unique entity in a knowledge base (KB) through entity disambigua- tion (ED)2. For example, in the sentence “The Times began publication under its current name in 1788,” the span The Times should be detected as a named entity mention and then linked to the corre- sponding entity: The Times, a British newspaper. However, an EL model which disjointly applies MD and ED might easily mistake this mention with The New York Times, an American newspa- per. Since our model jointly learns MD and ED from the same contextualized BERT embeddings, its final EL prediction is partially informed by both. As a result, it is able to generalize better. Another common approach employed in previ- ous EL research is candidate generation, where for each detected mention, a set of candidate entities is generated and the entities within it are ranked by a model to find the best match. Such sets are ∗Work done while at BlackRock. 1Also known as A2KB task in GERBIL evaluation plat- form (R¨oder et al., 2018) and end-to-end entity linking in some literature 2Also known as D2KB task in GERBIL built using hand-crafted rules which define which entities make it in and which do not. This risks (1) skipping out on valid entities which should be in the candidate set and (2) inflating model per- formance since often times candidate sets contain only one or two items. These sets are almost al- ways used at prediction time and sometimes even during training. Our model has the option of not relying on them during prediction, and never uses them during training. We introduce two main contributions: (i) We propose a new end-to-end differentiable neural EL model that jointly performs MD and ED and achieves state-of-the-art performance. (ii) We study the performance of our model when candidate sets are removed to see whether EL can perform well without them. 2 Related Work Neural-network based models have recently achieved strong results across standard EL datasets. Research has focused on learning bet- …

Content truncated to 500 words