Improving Entity Linking by Encoding Type Information into Entity Embeddings

Improving Entity Linking by Encoding Type Information into Entity Embeddings Tianran Li, Erguang Yang, Yujie Zhang†, Jinan Xu, Yufeng Chen School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China †yjzhang@bjtu.edu.cn Abstract Entity Linking (EL) refers to the task of linking entity mentions in the text to the correct entities in the Knowledge Base (KB) in which entity embeddings play a vital and challenging role because of the subtle differences between entities. However, existing pre-trained entity embeddings only learn the underlying semantic information in texts, yet the fine-grained entity type information is ignored, which causes the type of the linked entity is incompatible with the mention context. In order to solve this problem, we propose to encode fine-grained type information into entity embeddings. We firstly pre-train word vectors to inject type information by embedding words and fine-grained entity types into the same vector space. Then we retrain entity embeddings with word vectors containing fine-grained type information. By applying our entity embeddings to two existing EL models, our method respectively achieves 0.82% and 0.42% improvement on average F1 score of the test sets. Meanwhile, our method is model-irrelevant, which means it can help other EL models. 1 Introduction Entity Linking (EL) refers to the task of linking entity mentions in the text to correct entities in the Knowledge Base (KB) to help text comprehension by making use of rich semantic information in KB. The state-of-the-art entity linking models ignore fine-grained entity type information which causes the type of the linked entity is incompatible with the mention context. For example, among the error cases in AIDA-CoNLL(Hoffart et al., 2011) dataset produced by Le and Titov (2018), the mention “Japan” in the sentence “Japan began the defense of their Asian Cup” is linked to the entity “Japan”(country) while it should be linked to the entity “Japan national football team”(sports team). And such errors account for more than 1/2 of the total errors which show that their model cannot correctly distinguish different types of candidate entities. Entity embeddings are especially important to entity linking task for the sake of distinguishing entities with similar characters. We hope the entity embeddings can have type information so as to differentiate entities in terms of categories. However, previous pre-trained entity embeddings are learned by using plenty of texts, which ignores fine-grained type information (Yamada et al., 2016; Ganea and Hofmann, 2017). As a result, the similar entity vectors in the vector space are of different types. In order to alleviate this problem, we firstly pre-train word vectors to inject type information by embedding words and fine-grained entity types into the same vector space. Then, we follow Ganea and Hofmann (2017) to retrain the representation of each entity by making it closer to the word vectors of its context, using the word vectors containing type information. Finally, we apply the entity embeddings into the existing EL models without modifying the model architectures. Compared with two baseline models, our method respectively achieves 0.82% and 0.42% improvement on average …

Content truncated to 500 words