Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 583–595 August 1–6, 2021. ©2021 Association for Computational Linguistics 583 Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking Elliot Schumacher James Mayfield Johns Hopkins UniversityMark Dredze eschumac@cs.jhu.edu mayfield@jhu.edu mdredze@cs.jhu.edu Abstract Cross-language entity linking grounds men- tions written in several languages to a mono- lingual knowledge base. We use a simple neu- ral ranking architecture for this task that uses multilingual BERT representations of both the mention and the context as input, so as to explore the ability of a transformer model to perform well on this task. We find that the multilingual ability of BERT leads to good performance in monolingual and multilingual settings. Furthermore, we explore zero-shot language transfer and find surprisingly robust performance. We conduct several analyses to identify the sources of performance degra- dation in the zero-shot setting. Results indi- cate that while multilingual transformer mod- els transfer well between languages, issues re- main in disambiguating similar entities unseen in training. 1 Introduction Entity linking grounds named entities mentioned in text, such as Chancellor, to a reference knowl- edge base (KB) or ontology entry, such as Angela Merkel. Historically, entity linking work focused on English documents and knowledge bases, but subsequent work expanded the task to consider multiple languages (McNamee et al., 2011). In cross-language entity linking, entities in a set of multilingual documents is linked to a KB in a sin- gle language. The TAC KBP shared task (Ji et al., 2015), for example, links mentions in Chinese and Spanish documents with an English KB. Success in building cross-language linking systems can be helpful in tasks such as discovering all documents relevant to an entity, regardless of language. Successfully linking a mention across languages requires adapting several common entity linking components to the cross-language setting. Con- sider the example in Figure 1, which contains the Spanish mention Oficina de la Presidencia, a refer- ence to the entity President of Mexico in an English KB. To link the mention to the relevant entity we must compare the mention text and its surround- ing textual context in Spanish to the English entity name and entity description, as well as compare the mention and entity type. Previous work has focused on transliteration or translation approaches for name and context (McNamee et al., 2011; Pan et al., 2015), or leveraging large amounts of cross- language information (Tsai and Roth, 2016) and multilingual embeddings (Upadhyay et al., 2018). Since this work emerged, there have been ma- jor advances in multilingual NLP (Wu and Dredze, 2019; Pires et al., 2019). Mainstream approaches to multilingual learning now use multilingual en- coders, trained on raw text from multiple lan- guages (Devlin et al., 2019). These models, such as multilingual BERT or XMLR (Conneau et al., 2019), have achieved impressive results on a range of multilingual NLP tasks, including part of speech tagging (Tsai et al., 2019), parsing (Wang et al., 2019; Kondratyuk and Straka, 2019), and seman- tic similarity (Lo and Simard, 2019; Reimers and …

Content truncated to 500 words