Zero-Shot Entity Linking by Reading Entity Descriptions

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3449–3460 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3449 Zero-Shot Entity Linking by Reading Entity Descriptions Lajanugen Logeswaran†∗ Ming-Wei Chang‡ Kenton Lee‡ Kristina Toutanova‡ Jacob Devlin‡ Honglak Lee‡,† †University of Michigan, ‡Google Research {llajan,honglak}@umich.edu, {mingweichang,kentonl,kristout,jacobdevlin,honglak}@google.com Abstract We present the zero-shot entity linking task, where mentions must be linked to unseen en- tities without in-domain labeled data. The goal is to enable robust transfer to highly spe- cialized domains, and so no metadata or alias tables are assumed. In this setting, entities are only identified by text descriptions, and models must rely strictly on language under- standing to resolve the new entities. First, we show that strong reading comprehension mod- els pre-trained on large unlabeled data can be used to generalize to unseen entities. Second, we propose a simple and effective adaptive pre-training strategy, which we term domain- adaptive pre-training (DAP), to address the domain shift problem associated with linking unseen entities in a new domain. We present experiments on a new dataset that we construct for this task and show that DAP improves over strong pre-training baselines, including BERT. The data and code are available at https: //github.com/lajanugen/zeshel.1 1 Introduction Entity linking systems have achieved high per- formance in settings where a large set of dis- ambiguated mentions of entities in a target en- tity dictionary is available for training. Such systems typically use powerful resources such as a high-coverage alias table, structured data, and linking frequency statistics. For example, Milne and Witten (2008) show that by only using the prior probability gathered from hyperlink statis- tics on Wikipedia training articles, one can achieve 90% accuracy on the task of predicting links in Wikipedia test articles. While most prior works focus on linking to gen- eral entity databases, it is often desirable to link to ∗Work completed while interning at Google 1zeshel stands for zero-shot entity linking. Military Star Wars Elder Scrolls Burden ( Oblivion ) Burden is an Alteration spell that temporarily adds .. Burden ( Effect ) Burden is a spell effect that temporarily increases the weight.. Coronation Street Lego Mention All entities In the entity dictionary .... Orient Expedition Orient Expedition was one of the various subthemes... Orient Expedition Wallet Orient Expedition Wallet was a wallet themed around Orient Expedition.. .... Train Orient expedition is a train ride named after the theme of the same name. The train itself is .. Entity Linking Model Mention All entities In the entity dictionary Test The Burden spell is the opposite of Feather , increasing a character ' s encumbrance ... Figure 1: Zero-shot entity linking. Multiple training and test domains (worlds) are shown. The task has two key properties: (1) It is zero-shot, as no mentions have been observed for any of the test world entities during training. (2) Only textual (non-structured) information is available. specialized entity dictionaries such as legal cases, company project descriptions, the set of charac- ters …

Content truncated to 500 words