Distributional Thesauri for Information Retrieval and vice versa

Distributional Thesauri for Information Retrieval and vice versa Vincent Claveau1, Ewa Kijak2 1 IRISA-CNRS, 2 IRISA-University of Rennes 1 Campus de Beaulieu, 35042 Rennes, France vincent.claveau@irisa.fr ewa.kijak@irisa.fr Abstract Distributional thesauri are useful in many tasks of Natural Language Processing. In this paper, we address the problem of building and evaluating such thesauri with the help of Information Retrieval (IR) concepts. Two main contributions are proposed. First, following the work of (Claveau et al., 2014), we show how IR tools and concepts can be used with success to build a thesaurus. Through several experiments and by evaluating directly the results with reference lexicons, we show that some IR models outperform state-of-the-art systems. Secondly, we use IR as an applicative framework to indirectly evaluate the generated thesaurus. Here again, this task-based evaluation validates the IR approach used to build the thesaurus. Moreover, it allows us to compare these results with those from the direct evaluation framework used in the literature. The observed differences bring these evaluation habits into question. Keywords: distributional semantics, IR models, query expansion 1. Introduction Distributional semantics aims at building thesauri (or lex- icons) automatically from text corpora. For a given input (ie. a given word), these thesauri identify semantically sim- ilar words based on the assumption that they share a simi- lar distribution than the input word’s one. In practice, this distributional assumption is set such that two words would be considered close if their occurrences share similar con- texts. These contexts are typically co-occurring words in a limited window around the considered words, or words syntactically linked. Evaluating these thesauri remains a crucial point to as- sess the quality of the construction methods used. A com- monly used approach is to compare the generated thesauri to one or several reference lexicons. This evaluation proce- dure, called ’intrinsic’, has the advantage of being straight- forward and simple as it allows to estimate the quality and completeness of the generated thesaurus. However, it is based on reference lexicons whose own completeness, quality, or simply their availability for the considered do- main/language/genre are not always granted. In this article1, we propose to examine those two aspects – the construction and the evaluation of distributional the- sauri – by using information retrieval (IR) both as a set of techniques and as a use case. Concerning the construction, recent work (Claveau et al., 2014) showed that IR systems could advantageously be used to implement distributional analysis systems. We propose in this paper to further ex- plore this IR approach to build thesauri. We examine the interest of various classic models of IR for distributional analysis and compare them with the state-of-the-art. Regarding the evaluation, we offer an extrinsic evaluation of the generated thesauri through a conventional IR task. We are then able to compare these results with those of the intrinsic evaluation, and therefore to judge the relevance of 1This work was partly funded via the BigClin and LIMAH CominLabs excellence laboratory projects financed by the French National Research Agency under reference …

Content truncated to 500 words