Nothing like Good Old Frequency: Studying Context Filters for Distributional Thesauri

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 419–424, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Nothing like Good Old Frequency: Studying Context Filters for Distributional Thesauri Muntsa Padr´o,♣ Marco Idiart♥, Carlos Ramisch♦, Aline Villavicencio♣ ♣Institute of Informatics, Federal University of Rio Grande do Sul (Brazil) ♥Institute of Physics, Federal University of Rio Grande do Sul (Brazil) ♦Aix Marseille Universit´e, CNRS, LIF UMR 7279, 13288, Marseille (France) muntsa.padro@inf.ufrgs.br, marco.idiart@gmail.com, carlos.ramisch@lif.univ-mrs.fr, avillavicencio@inf.ufrgs.br Abstract Much attention has been given to the impact of informativeness and similar- ity measures on distributional thesauri. We investigate the effects of context fil- ters on thesaurus quality and propose the use of cooccurrence frequency as a sim- ple and inexpensive criterion. For eval- uation, we measure thesaurus agreement with WordNet and performance in answer- ing TOEFL-like questions. Results illus- trate the sensitivity of distributional the- sauri to filters. 1 Introduction Large-scale distributional thesauri created auto- matically from corpora (Grefenstette, 1994; Lin, 1998; Weeds et al., 2004; Ferret, 2012) are an inexpensive and fast alternative for representing semantic relatedness between words, when man- ually constructed resources like WordNet (Fell- baum, 1998) are unavailable or lack coverage. To construct a distributional thesaurus, the (colloca- tional or syntactic) contexts in which a target word occurs are used as the basis for calculating its sim- ilarity with other words. That is, two words are similar if they share a large proportion of contexts. Much attention has been devoted to refin- ing thesaurus quality, improving informativeness and similarity measures (Lin, 1998; Curran and Moens, 2002; Ferret, 2010), identifying and de- moting bad neighbors (Ferret, 2013), or using more relevant contexts (Broda et al., 2009; Bie- mann and Riedl, 2013). For the latter in particular, as words vary in their collocational tendencies, it is difficult to determine how informative a given context is. To remove uninformative and noisy contexts, filters have often been applied like point- wise mutual information (PMI), lexicographer’s mutual information (LMI) (Biemann and Riedl, 2013), t-score (Piasecki et al., 2007) and z-score (Broda et al., 2009). However, the selection of a measure and of a threshold value for these filters is generally empirically determined. We argue that these filtering parameters have a great influence on the quality of the generated thesauri. The goal of this paper is to quantify the im- pact of context filters on distributional thesauri. We experiment with different filter methods and measures to assess context significance. We pro- pose the use of simple cooccurrence frequency as a filter and show that it leads to better results than more expensive measures such as LMI or PMI. Thus we propose a cheap and effective way of fil- tering contexts while maintaining quality. This paper is organized as follows: in §2 we discuss evaluation of distributional thesauri. The methodology adopted in the work and the results are discussed in §3 and §4. We finish with some conclusions and discussion of future work. 2 Related Work In a nutshell, the standard …

Content truncated to 500 words