Learning Dense Representations for Entity Retrieval

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 528–537 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 528 Learning Dense Representations for Entity Retrieval Daniel Gillick∗ Google Research dgillick@google.com Sayali Kulkarni∗ Google Research sayali@google.com Larry Lansing∗ Google Research llansing@google.com Alessandro Presta∗ Google Research apresta@google.com Jason Baldridge Google Research jasonbaldridge@google.com Eugene Ie Google Research eugeneie@google.com Diego Garcia-Olano† University of Texas at Austin diegoolano@gmail.com Abstract We show that it is feasible to perform entity linking by training a dual encoder (two-tower) model that encodes mentions and entities in the same dense vector space, where candidate entities are retrieved by approximate nearest neighbor search. Unlike prior work, this setup does not rely on an alias table followed by a re-ranker, and is thus the first fully learned en- tity retrieval model. We show that our dual encoder, trained using only anchor-text links in Wikipedia, outperforms discrete alias ta- ble and BM25 baselines, and is competitive with the best comparable results on the stan- dard TACKBP-2010 dataset. In addition, it can retrieve candidates extremely fast, and gen- eralizes well to a new dataset derived from Wikinews. On the modeling side, we demon- strate the dramatic value of an unsupervised negative mining algorithm for this task. 1 Introduction A critical part of understanding natural language is con- necting specific textual references to real world entities. In text processing systems, this is the task of entity res- olution: given a document where certain spans of text have been recognized as mentions referring to entities, the goal is to link them to unique entries in a knowledge base (KB), making use of textual context around the mentions as well as information about the entities. (We use the term mention to refer to the target span along with its context in the document.) Real world knowledge bases are large (e.g., English Wikipedia has 5.7M articles), so existing work in entity resolution follows a two-stage approach: a first compo- nent nominates candidate entities for a given mention and a second one selects the most likely entity among those candidates. This parallels typical information re- trieval systems that consist of an index and a re-ranking model. In entity resolution, the index is a table mapping aliases (possible names) to entities. Such tables need to be built ahead of time and are typically subject to ∗Equal Contributions †Work done during internship with Google arbitrary, hard cutoffs, such as only including the thirty most popular entities associated with a particular men- tion. We show that this configuration can be replaced with a more robust model that represents both entities and mentions in the same vector space. Such a model allows candidate entities to be directly and efficiently retrieved for a mention, using nearest neighbor search. To see why a retrieval approach is desirable, we need to consider how alias tables are employed in entity reso- lution systems. In the following example, Costa refers to footballer Jorge Costa, but the entities associated with that alias in existing …

Content truncated to 500 words