WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4785 - 4797 May 22-27, 2022 c⃝2022 Association for Computational Linguistics WIKIDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types Xuwu Wang1, Junfeng Tian2, Min Gui3∗, Zhixu Li1�, Rui Wang4, Ming Yan2, Lihan Chen1, Yanghua Xiao1,5� 1 School of Computer Science, Fudan University, China 2 Alibaba Group, China 3 Shopee, Singapore 4 Vipshop (China) Co., Ltd., China 5 Fudan-Aishu Cognitive Intelligence Joint Research Center, China {xwwang18,zhixuli,shawyh}@fudan.edu.cn, {tjf141457, ym119608}@alibaba-inc.com, min.gui@shopee.com, mars198356@hotmail.com, lhc825@gmail.com Abstract Multimodal Entity Linking (MEL) which aims at linking mentions with multimodal con- texts to the referent entities from a knowl- edge base (e.g., Wikipedia), is an essen- tial task for many multimodal applications. Although much attention has been paid to MEL, the shortcomings of existing MEL datasets including limited contextual topics and entity types, simplified mention ambigu- ity, and restricted availability, have caused great obstacles to the research and appli- cation of MEL. In this paper, we present WIKIDiverse, a high-quality human-annotated MEL dataset with diversified contextual topics and entity types from Wikinews, which uses Wikipedia as the corresponding knowledge base. A well-tailored annotation procedure is adopted to ensure the quality of the dataset. Based on WIKIDiverse, a sequence of well- designed MEL models with intra-modality and inter-modality attentions are implemented, which utilize the visual information of im- ages more adequately than existing MEL mod- els do. Extensive experimental analyses are conducted to investigate the contributions of different modalities in terms of MEL, facili- tating the future research on this task. The dataset and baseline models are available at https://github.com/wangxw5/wikiDiverse. 1 Introduction Entity linking (EL) has attracted increasing atten- tion in the natural language processing community, which aims at linking ambiguous mentions to the referent unambiguous entities in a given knowledge base (KB) (Shen et al., 2014). It has been applied to a lot of downstream tasks such as information extraction (Yaghoobzadeh et al., 2016), question ∗This work was conducted when Min Gui worked at Alibaba. Match of the 50m freestyle final Freestyle_ swimming Freestyle rap FORD freestyle ✅ The Lions versus the Packers. Detroit Lions (American football team) London Lions (basketball team) Mac OS X Lion (Operating system) ✅ The ISS from Atlantis Space Shuttle Atlantis Atlantis Atlantis (2011 film) ✅ Contexts of the Mention Contexts of the Candidate Entities Figure 1: Several MEL examples with mentions high- lighted in the caption and the first entity of each entity listed as the gold label. answering (Yih et al., 2015) and semantic search (Blanco et al., 2015). As named entities (i.e., mentions) with multi- modal contexts such as texts and images are ubiq- uitous in daily life, recent studies (Moon et al., 2018; Adjali et al., 2020a) turn their focus towards improving the performance of EL models through utilizing visual information, i.e., Multimodal En- tity linking (MEL)1. Several MEL examples are depicted in Figure 1, where the images could effec- tively help the disambiguation for entity mentions …

Content truncated to 500 words