Low-Rank Subspaces for Unsupervised Entity Linking

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8037–8054 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8037 Low-Rank Subspaces for Unsupervised Entity Linking Akhil Arora EPFL akhil.arora@epfl.ch Alberto García-Durán∗ Atinary Technologies agaduran@gmail.com Robert West EPFL robert.west@epfl.ch Abstract Entity linking is an important problem with many applications. Most previous solutions were designed for settings where annotated training data is available, which is, however, not the case in numerous domains. We pro- pose a light-weight and scalable entity linking method, EIGENTHEMES, that relies solely on the availability of entity names and a referent knowledge base. EIGENTHEMES exploits the fact that the entities that are truly mentioned in a document (the “gold entities”) tend to form a semantically dense subset of the set of all can- didate entities in the document. Geometrically speaking, when representing entities as vectors via some given embedding, the gold entities tend to lie in a low-rank subspace of the full embedding space. EIGENTHEMES identifies this subspace using the singular value decom- position and scores candidate entities accord- ing to their proximity to the subspace. On the empirical front, we introduce multiple strong baselines that compare favorably to (and some- times even outperform) the existing state of the art. Extensive experiments on benchmark datasets from a variety of real-world domains showcase the effectiveness of our approach. 1 Introduction Entity linking (EL) is the task of grounding men- tions to a reference knowledge base (also referred to as knowledge graph). With a plethora of applica- tions, including but not limited to information ex- traction (Hoffart et al., 2011) and automatic knowl- edge base construction (Gao et al., 2018), EL is one of the most actively researched topics in natural lan- guage processing. Despite the recent proliferation of EL methods, recent works (Tsai and Roth, 2016; Upadhyay et al., 2018) have pointed out that the performance of existing techniques largely relies on the existence of large corpora of annotated data. ∗Research done while at EPFL. Unsupervised entity linking. In contrast, truly un- supervised entity linkers should be able to operate in the absolute absence of annotated data, with ac- cess to only a reference knowledge graph (KG) and a list of entity names, or aliases. Furthermore, nei- ther the candidate generator nor the disambiguation technique (the two key modules of any typical en- tity linker) can make use of annotated data. There- fore, “unsupervised” disambiguation techniques that leverage labeled data to generate candidate en- tities (Pan et al., 2015) are not applicable in our setting. Only very recently have researchers (Le and Titov, 2019; Logeswaran et al., 2019) started to focus on EL systems that can operate in the ab- solute absence of annotated data. The motivation is well founded: there are some domains that link to their own specialized KGs, for which labeled data is not readily available, expensive to obtain, or scarce in the best case. This can be exemplified with domains such as law, science, or medicine. Moreover, companies may have their …

Content truncated to 500 words