Towards Better Entity Linking with Multi-View Enhanced Distillation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9729–9743 July 9-14, 2023 ©2023 Association for Computational Linguistics Towards Better Entity Linking with Multi-View Enhanced Distillation Yi Liu1,2,∗, Yuan Tian3, Jianxun Lian3, Xinlong Wang3, Yanan Cao1,2,†, Fang Fang1,2, Wen Zhang3, Haizhen Huang3, Denvy Deng3, Qi Zhang3 1Institute of Information Engineering, Chinese Academy of Sciences 2School of Cyber Security, University of Chinese Academy of Sciences 3Microsoft {liuyi1999,caoyanan,fangfang0703}@iie.ac.cn {yuantian,jialia,xinlongwang,zhangw,hhuang,dedeng,qizhang}@microsoft.com Abstract Dense retrieval is widely used for entity linking to retrieve entities from large-scale knowledge bases. Mainstream techniques are based on a dual-encoder framework, which encodes men- tions and entities independently and calculates their relevances via rough interaction metrics, resulting in difficulty in explicitly modeling multiple mention-relevant parts within entities to match divergent mentions. Aiming at learn- ing entity representations that can match diver- gent mentions, this paper proposes a Multi- View Enhanced Distillation (MVD) frame- work, which can effectively transfer knowledge of multiple fine-grained and mention-relevant parts within entities from cross-encoders to dual-encoders. Each entity is split into mul- tiple views to avoid irrelevant information be- ing over-squashed into the mention-relevant view. We further design cross-alignment and self-alignment mechanisms for this framework to facilitate fine-grained knowledge distillation from the teacher model to the student model. Meanwhile, we reserve a global-view that em- beds the entity as a whole to prevent dispersal of uniform information. Experiments show our method achieves state-of-the-art performance on several entity linking benchmarks1. 1 Introduction Entity Linking (EL) serves as a fundamental task in Natural Language Processing (NLP), connect- ing mentions within unstructured contexts to their corresponding entities in a Knowledge Base (KB). EL usually provides the entity-related data foun- dation for various tasks, such as KBQA (Ye et al., 2022), Knowledge-based Language Models (Liu et al., 2020) and Information Retrieval (Li et al., 2022). Most EL systems consist of two stages: en- tity retrieval (candidate generation), which retrieves *Work is done during internship at Microsoft. †Corresponding Author. 1Our code is available at https://github.com/Noen61/ MVD Entity 1: 2014 UEFA Champions League final Description: Real Madrid won the match 4–1 after extra time, with goals from Cristiano Ronaldo, Gareth Bale, Marcelo and Sergio Ramos. In doing so, Real Madrid secured a record 10th title in the competition. As the winners, Real Madrid earned the right to play against 2013–14 UEFA Europa League winners Sevilla in the 2014 UEFA Super Cup. Mention: Ronaldo fired home the penalty as Real Madrid won Europe’s biggest prize for the 10th time in its history. Entity 2: Cristiano Ronaldo Description: …Ronaldo has won five Ballon d'Or awards and four European Golden Shoes, he has won 32 trophies in his career, including seven league titles, five UEFA Champions Leagues, the UEFA European Championship and the UEFA Nations League. ... Ronaldo was cautioned by police for smashing a phone out of a 14-year-old boy’s hand following his team’s 1-0 Premier League defeat to Everton in April. Mention: Ronaldo has amassed an unrivalled collection of records in the Champions League …

Content truncated to 500 words