A Transformational Biencoder with In-Domain Negative Sampling for Zero-Shot Entity Linking

Findings of the Association for Computational Linguistics: ACL 2022, pages 1449 - 1458 May 22-27, 2022 c⃝2022 Association for Computational Linguistics A Transformational Biencoder with In-Domain Negative Sampling for Zero-Shot Entity Linking Kai Sun,1 Richong Zhang,1∗Samuel Mensah,2 Yongyi Mao,3 Xudong Liu1 1SKLSDE, Beihang University, Beijing, China 2Department of Computer Science, University of Sheffield, UK 3 School of Electrical Engineering and Computer Science, University of Ottawa, Canada sunkai@buaa.edu.cn, s.mensah@sheffield.ac.uk, ymao@uottawa.ca zhangrc,liuxd@act.buaa.edu.cn Abstract Recent interest in entity linking has focused in the zero-shot scenario, where at test time the en- tity mention to be labelled is never seen during training, or may belong to a different domain from the source domain. Current work leverage pre-trained BERT has the implicit assumption that BERT bridges the gap between the source and target domain distributions. However, fine- tuned BERT has a considerable underperfor- mance at zero-shot when applied in a different domain. We solve this problem by proposing a Transformational Biencoder that incorporates a transformation into BERT to perform a zero- shot transfer from the source domain during training. As like previous work, we rely on negative entities to encourage our model to dis- criminate the golden entities during training. To generate these negative entities, we propose a simple but effective strategy that takes the do- main of the golden entity into perspective. Our experimental results on the benchmark dataset Zeshel show effectiveness of our approach and achieve new state-of-the-art. 1 Introduction Entity Linking (EL) is an important task in Natural Language Processing (NLP), which seeks to align entity mentions in a document to their referent en- tity in a knowledge base such as Wikipedia. EL has received widespread attention due to its appli- cation in a variety of tasks, including information extraction (Lin et al., 2012), knowledge base popu- lation (Dredze et al., 2010), content analysis (Weng et al., 2010), etc. There has been great achieve- ment in building EL systems, however, majority of proposed works (Ganea and Hofmann, 2017; Cao et al., 2018) are built on the assumption that the entity set is shared among the train and test sets. In many practical cases, however, the train and test sets may come from different domain distributions. This potentially creates disjoint entity sets across ∗Corresponding author the different domains, highlighting the importance of zero-shot EL (Sil et al., 2012; Logeswaran et al., 2019). Zero-shot EL aims to label mentions in the test set that have never been seen during training. A line of works have proposed zero-shot learning techniques for entity linking (Sil et al., 2012; Wang et al., 2015). The common paradigm in these works is to link labelled mentions in a document to enti- ties in well structured knowledge bases. Despite their promising successes, labelled data is typically expensive to produce or are not easily obtained for some specialized domains such as the legal domain. To enable research in this problem, Lo- geswaran et al. (2019) developed the Zeshel dataset which contains a diverse range of specialized do- mains, in which mentions …

Content truncated to 500 words