Collective Search for Concept Disambiguation

Proceedings of COLING 2012: Technical Papers, pages 2243–2258, COLING 2012, Mumbai, December 2012. Collective Search for Concept Disambiguation Anja PI LZ Gerhard PAASS Fraunhofer IAIS, Schloss Birlinghoven, 53757 Sankt Augustin, Germany anja.pilz@iais.fraunhofer.de, gerhard.paass@iais.fraunhofer.de ABSTRACT Name ambiguity is a major problem in information retrieval: The name "Metropolis" may refer to a movie, a physicist, or Superman’s hometown. Recent work resolves ambiguity in natural language text by linking name mentions against the corresponding Wikipedia concept (Wikification). Standard methods comparing a single mention with the corresponding Wikipedia concept can potentially be improved by simultaneously considering all mentions in the input document. We propose a novel multiple assignment process based on a collective search over an inverted index that exploits the coherence of Wikipedia concepts. Based on this coherence, we compute the best fitting candidate concept for each mention and combine it with context information in a second search step. Using additional attributes an SVM then re-ranks the result of this search and estimates if a concept is not covered in Wikipedia. We give a unified view over the different performance measures used in other state-of-the art approaches and evaluate our approach on five benchmark corpora. On these corpora, our method has the most stable performance yielding similar or better results compared to other approaches. KEYWORDS: Concept and Entity Disambiguation, Wikification, Natural Language Processing, Search and Ranking. 2243 1 Introduction A major aim of search engines is the retrieval of information about concepts which may be any existing object, e.g. person, thing, notion, etc., with a designation or name. In natural language text however, many concepts share the same name and one concept may be referenced by different names. Consequently, a search based on pure string matching often yields many irrelevant results, such as a web page on Superman’s hometown when indeed the user seeked information on the physicist Metropolis. Concept disambiguation which assigns the correct sense to the mention of a concept in a given context, can reduce the number of irrelevant results or group results by sense. The disambiguation of concept mentions is required in applications such as semantic search, but also many other areas like knowledge base construction or data base curation. Recent work, for example (Ratinov et al., 2011), resolves name ambiguity by linking the name mention against the corresponding Wikipedia article, thus often terming the problem Wikification. For that, a name mention together with the features of its neighboring context is compared to the corresponding features of the Wikipedia article. If the difference between these features is small, a Wikipedia concept is linked to the mention and thus the name’s ambiguity considered as resolved. A large number of features have been evaluated for concept disambiguation. Starting with simple bag-of-word descriptions more advanced features were developed characterizing the sense of surrounding words, e.g. topic model indices (Pilz and Paass, 2011). But often, approaches remained local and did not exploit the global coherence of candidate concepts. In this paper we follow the global approach by simultaneously considering all mentions of an …

Content truncated to 500 words