Multilingual Autoregressive Entity Linking

Multilingual Autoregressive Entity Linking Nicola De Cao1,4, Ledell Wu7, Kashyap Popat1, Mikel Artetxe1, Naman Goyal3, Mikhail Plekhanov1, Luke Zettlemoyer3,5, Nicola Cancedda1, Sebastian Riedel1,6, Fabio Petroni1 1Facebook AI, UK 2Facebook AI, China 3Facebook AI, USA 4University of Amsterdam, NL 5University of Washington, USA 6University College London, UK 7Beijing Academy of Artificial Intelligence, China nicola.decao@gmail.com, {ledell, kpopat, artetxe, naman, movb, lsz, ncan, sriedel, fabiopetroni}@fb.com, wuyu.ledell@gmail.com Abstract We present mGENRE, a sequence-to- sequence system for the Multilingual En- tity Linking (MEL) problem—the task of resolving language-specific mentions to a multilingual Knowledge Base (KB). For a mention in a given language, mGENRE pre- dicts the name of the target entity left-to-right, token-by-token in an autoregressive fashion. The autoregressive formulation allows us to effectively cross-encode mention string and entity names to capture more interactions than the standard dot product between mention and entity vectors. It also enables fast search within a large KB even for mentions that do not appear in mention tables and with no need for large-scale vector indices. While prior MEL works use a single representation for each entity, we match against entity names of as many languages as possible, which allows exploiting language connections between source input and target name. Moreover, in a zero-shot setting on languages with no training data at all, mGENRE treats the target language as a latent variable that is marginalized at prediction time. This leads to over 50% improvements in average accuracy. We show the efficacy of our approach through extensive evaluation including experiments on three popular MEL benchmarks where we establish new state-of-the-art results. Source code available at https://github.com /facebookresearch/GENRE. 1 Introduction Entity Linking (EL, Bunescu and Pas¸ca, 2006; Cucerzan, 2007; Hoffart et al., 2011; Dredze et al., 2010) is an important task in NLP, with plenty of applications in multiple domains, spanning Ques- tion Answering (De Cao et al., 2019; Nie et al., 2019; Asai et al., 2020), Dialogue (Bordes et al., 2017; Wen et al., 2017; Williams et al., 2017; Chen et al., 2017; Curry et al., 2018; Sevegnani et al., 2021), and Biomedical systems (Leaman and Gonzalez, 2008; Zheng et al., 2015), to name just a few. It consists of grounding entity mentions in unstructured texts to KB descriptors (e.g., Wiki- pedia articles). The multilingual version of the EL problem has been for a long time tied to a purely cross-lingual formulation (XEL, McNamee et al., 2011; Ji et al., 2015), where mentions expressed in one language are linked to a KB expressed in another (typically English). Recently, Botha et al. (2020) made a step towards a more inherently multilin- gual formulation by defining a language-agnostic KB, obtained by grouping language-specific de- scriptors per entity. Such a formulation has the power of considering entities that do not have an English descriptor (e.g., a Wikipedia article in English) but have one in some other languages. A common design choice to most current solu- tions, regardless of the specific formulation, is to provide a unified entity representation, either by collating multilingual descriptors in a …

Content truncated to 500 words