Combining Open Source Annotators for Entity Linking through Weighted Voting

Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM 2015), pages 211–215, Denver, Colorado, June 4–5, 2015. Combining Open Source Annotators for Entity Linking through Weighted Voting Pablo Ruiz and Thierry Poibeau LATTICE Lab CNRS, École Normale Supérieure, U Paris 3 Sorbonne Nouvelle 1, rue Maurice Arnoux, 92120 Montrouge, France {pablo.ruiz.fabo,thierry.poibeau}@ens.fr Abstract An English entity linking (EL) workflow is presented, which combines the annotations of five public open source EL services. The an- notations are combined through a weighted voting scheme inspired by the ROVER meth- od, which had not been previously tested on EL outputs. The combined results improved over each individual system’s results, as eval- uated on four different golden sets. 1 Introduction The Entity Linking (EL) literature has shown that the quality of EL systems’ results varies widely depending on characteristic of the corpora they are applied to, or on the types of entities we need to link (Cornolti et al., 2013, Usbeck et al., 2015). For instance, a system that links to a wide set of entity types can be less accurate at basic types like Per- son, Location, Organization than systems special- izing in those basic types. A way to make up for the uneven performance of entity linking methods across corpora would be mixing different annotators’ results, so that the annotators’ strengths complement each other. This paper presents a method to combine the outputs of five open source entity linking systems, in order to obtain improved results. The method involves a weighted voting scheme that had not been previ- ously applied to EL, and improves annotation re- sults across four test-corpora. The structure of the paper is as follows: Section 2 presents related work. Section 3 describes the combined entity linking system. Section 4 provides an evaluation of the system’s results and a discus- sion. 2 Related Work General surveys on EL can be found in (Cornolti et al., 2013) and (Rao et al., 2013). Besides the EL literature, work on combining NLP annotators is particularly relevant for the present article. The goal of combining different NLP systems is obtaining combined results that are better than the results of each individual system. Fiscus (1997) created the ROVER method, with weighted voting to improve speech recognition outputs. ROVER was found to improve parsing results by De la Clergerie et al. (2008). In Named Entity Recogni- tion (NER), Rizzo et al. (2014) improved results combining systems via different machine learning algorithms. In entity linking, the potential benefits of com- bining annotations have been explored before. Riz- zo and Troncy (2012) describe the NERD system, which combines entity linkers. However, we are not aware of a system that, like ours, makes an au- tomatic choice among the systems’ conflicting an- notations, based on an estimate of each annotation’s quality. Our approach to choose among conflicting annotations is inspired by the ROVER method, which had not been previously attempted for EL to our knowledge. A further dif- ference in our system is that the set …

Content truncated to 500 words