EL92: Entity Linking Combining Open Source Annotators via Weighted Voting

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 355–359, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics EL92: Entity Linking Combining Open Source Annotators via Weighted Voting Pablo Ruiz and Thierry Poibeau Laboratoire LATTICE CNRS, École Normale Supérieure, U. Paris 3 Sorbonne Nouvelle 1, rue Maurice Arnoux 92120 Montrouge, France {pablo.ruiz.fabo,thierry.poibeau}@ens.fr Abstract Our participation at SemEval’s Multilingual All-Words Sense Disambiguation and Entity Linking task is described. An English entity linking (EL) system is presented, which com- bines the annotations of four public open source EL services. The annotations are com- bined through a weighted voting scheme in- spired on the ROVER method, which had not been previously tested on EL outputs. Results on the task’s EL items were competitive. 1 Introduction The paper describes our participation at SemEval 2015, Task 13 (Moro and Navigli, 2015): Multilin- gual all-words Sense Disambiguation (WSD) and Entity Linking (EL). Systems performing both tasks, or either one, can participate. The preferred word-sense and entity inventory is Babelnet (Navi- gli and Ponzetto, 2012); other inventories are al- lowed. Our system performs English EL to Wikipedia, combining the output of open-source, publicly available EL systems via weighted voting. The system is relevant to the task’s interest in comparing the results of EL systems that apply encyclopedic knowledge only, like ours, and sys- tems that jointly exploit encyclopedic and lexico- graphic resources for EL. The paper’s structure is the following: Section 2 discusses related work, and Section 3 describes the system. Sections 4 and 5 present the results and a conclusion. 2 Related Work General surveys on EL can be found in (Cornolti et al., 2013) and (Rao et al., 2013). Work on combin- ing NLP annotators and on evaluating EL systems is particularly relevant for our submission. The goal of combining different NLP systems is obtaining combined results that are better than the results of each individual system. Fiscus (1997) created the ROVER method, with weighted voting to improve speech recognition outputs. A ROVER was found to improve parsing results by De la Clergerie et al. (2008). Rizzo et al. (2014) im- proved Named Entity Recognition results, combin- ing systems via different machine learning algorithms. Our approach is inspired on the ROVER method, which had not been previously attempted for EL to our knowledge. Systems that combine entity linkers exist (NERD, Rizzo and Troncy, 2012). However, a difference in our sys- tem is that the set of linkers we combine is public and open-source. A second difference is the set of methods we employed to combine annotations. EL evaluation work (Cornolti et al., 2013), (Usbeck et al., 2015) has highlighted to what an extent EL systems’ performance can differ depend- ing on characteristics of the corpus. This motivates testing whether different EL systems, properly combined, can complement each other. 355 3 System Description The system performs English EL to Wikipedia, combining the outputs of the following EL sys- tems: Tagme 21 (Ferragina and Scaiella, 2010), DBpedia Spotlight2 (Mendes et al. 2011), …

Content truncated to 500 words