MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2617–2624 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2617 MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations Xinyin Ma⋄‡, Yong Jiang†∗, Nguyen Bach†, Tao Wang†, Zhongqiang Huang†, Fei Huang†, Weiming Lu⋄∗ ⋄ College of Computer Science and Technology , Zhejiang University † DAMO Academy, Alibaba Group {maxinyin, luwm}@zju.edu.cn, yongjiang.jy@alibaba-inc.com Abstract Entity retrieval, which aims at disambiguating mentions to canonical entities from massive KBs, is essential for many tasks in natural lan- guage processing. Recent progress in entity retrieval shows that the dual-encoder structure is a powerful and efficient framework to nomi- nate candidates if entities are only identified by descriptions. However, they ignore the prop- erty that meanings of entity mentions diverge in different contexts and are related to vari- ous portions of descriptions, which are treated equally in previous works. In this work, we propose Multi-View Entity Representations (MuVER), a novel approach for entity retrieval that constructs multi-view representations for entity descriptions and approximates the opti- mal view for mentions via a heuristic search- ing method. Our method achieves the state-of- the-art performance on ZESHEL and improves the quality of candidates on three standard En- tity Linking datasets1. 1 Introduction Entity linking (EL) refers to the task that disam- biguates the mentions in textual input and retrieves the corresponding unique entity in large Knowl- edge Bases (KBs) (Han et al., 2011; Ceccarelli et al., 2013). The majority of neural entity retrieval approaches consist of two steps: Candidate Gen- eration (Pershina et al., 2015; Zwicklbauer et al., 2016), which nominates a small list of candidates from millions of entities with low-latency algo- rithms, and Entity Ranking (Yang et al., 2018; Le and Titov, 2019; Cao et al., 2021), which ranks those candidates to select the best match with more sophisticated algorithms. In this paper, we focus on the Candidate Genera- tion problem (a.k.a. the first-stage retrieval). Prior ∗Corresponding authors. ‡ Work was done when Xinyin Ma was interning at Ali- baba DAMO Academy. 1Our source code is available at https://github. com/Alibaba-NLP/MuVER. works filter entities by alias tables (Fang et al., 2019) or precalculated mention-entity prior prob- abilities, e.g., p(entity|mention) (Le and Titov, 2018). Ganea and Hofmann (2017) and Yamada et al. (2016) build entity embedding from the local context of hyperlinks in entity pages or entity-entity co-occurrences. Those embedding-based methods were extended by BLINK (Wu et al., 2020) and DEER (Gillick et al., 2019) to two-tower dual- encoders (Khattab and Zaharia, 2020), which en- code mentions and descriptions of entities into high- dimensional vectors respectively. Candidates are retrieved by nearest neighbor search (Andoni and Indyk, 2008; Johnson et al., 2019) for a given men- tion. Solutions that require only entity descriptions (Logeswaran et al., 2019) are scalable, as descrip- tions are more readily obtainable than statistical or manually annotated resources. Although description-based dual-encoders can compensate for the weakness of traditional meth- ods and have better generalization ability to unseen domains, they aim to map mentions …

Content truncated to 500 words