Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 171–180 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 171 Learning text representations for 500K classification tasks on Named Entity Disambiguation Ander Barrena and Aitor Soroa and Eneko Agirre IXA NLP Group UPV/EHU University of the Basque Country Donostia, Basque Country ander.barrena,a.soroa,e.agirre@ehu.eus Abstract Named Entity Disambiguation algorithms typ- ically learn a single model for all target en- tities. In this paper we present a word ex- pert model and train separate deep learning models for each target entity string, yielding 500K classification tasks. This gives us the opportunity to benchmark popular text repre- sentation alternatives on this massive dataset. In order to face scarce training data we pro- pose a simple data-augmentation technique and transfer-learning. We show that bag- of-word-embeddings are better than LSTMs for tasks with scarce training data, while the situation is reversed when having larger amounts. Transferring an LSTM which is learned on all datasets is the most effective context representation option for the word ex- perts in all frequency bands. The experi- ments show that our system trained on out-of- domain Wikipedia data surpasses comparable NED systems which have been trained on in- domain training data. 1 Introduction Named Entity Disambiguation (NED), also known as Entity Linking or Entity Resolution, is a task where entity mentions in running text need to be linked to its entity entry in a Knowledge Base (KB), such as Wikidata, Wikipedia or other de- rived resources like DBpedia (Bunescu and Pasca, 2006; McNamee and Dang, 2009; Hoffart et al., 2011). This task is challenging, as some en- tity mentions like “London” can refer to a num- ber of places, people, fictional characters, brands, movies, books or songs. Given a mention in context, NED methods (Cucerzan, 2007; Han and Sun, 2011; Ratinov et al., 2011; Lazic et al., 2015) typically rely on three models: (1) a mention model which collects possible entities which can be referred to by the mention string (aliases or surface forms), possi- bly weighted according to prior probabilities; (2) a context model which measures to which extent the entities fit well in the context of the mention, us- ing textual features; (3) a coherence model which prefers entities that are related to the other entities in the document. The first and second models are local in that they only require a short context of occurrence and disambiguate each mention in the document separately. The third model is global, in that all mentions are disambiguated simulta- neously (Ratinov et al., 2011). Recent work has shown that local models can be improved adding a global coherence model (Ratinov et al., 2011; Globerson et al., 2016). In this work we focus on a local model, and a global model could improve the results further. All local and global systems mentioned above, as well as the current state-of-the-art systems (Lazic et al., 2015; Globerson et al., 2016; Yamada et al., 2016; Ganea and Hofmann, …

Content truncated to 500 words