Singular Value Decomposition for Feature Selection in Taxonomy Learning

International Conference RANLP 2009 - Borovets, Bulgaria, pages 82–87 Singular Value Decomposition for Feature Selection in Taxonomy Learning Francesca Fallucchi and Fabio Massimo Zanzotto ∗ Via del Politecnico 00133 Rome, Italy {fallucchi,zanzotto}@info.uniroma2.it Abstract In this paper, we propose a novel way to include unsupervised feature selection meth- ods in probabilistic taxonomy learning mod- els. We leverage on the computation of logistic regression to exploit unsupervised feature se- lection of singular value decomposition (SVD). Experiments show that this way of using SVD for feature selection positively affects perfor- mances. 1 Introduction Taxonomies and, in general, networks of words con- nected with transitive relations are extremely impor- tant knowledge repositories for a variety of applica- tions in natural language processing (NLP) and knowl- edge representation (KR). In NLP, taxonomies such as WordNet [17] are widely used in intermediate tasks such as word sense disambiguation (e.g. [1]) and se- lectional preference induction (e.g., [25]) as well as in final applications such as question answering (e.g., [4]) and textual entailment recognition (e.g. [5]). In KR, taxonomies as well as other word networks are the bulk of domain ontologies. To be effectively used in NLP and KR applica- tions, taxonomies and knowledge repositories have to be large or, at least, adapted to specific domains. Yet, even huge knowledge repositories such as WordNet [17] are extremely poor when used in specific domains such as the medical domain (see [29]). Automatically creat- ing, adapting, or extending existing knowledge repos- itories using domain texts is, then, a very important and active area. A large variety of methods have been proposed: ontology learning methods [16, 3, 19] in KR as well as knowledge harvesting methods in NLP such as [13, 21]. These learning methods use variants of the distributional hypothesis [12] or exploit some induced lexical-syntactic patterns (originally used in [26]). The task is generally seen as a classification (e.g., [22, 27]) or a clustering (e.g., [3]) problem. This allows the use of machine learning models. Yet, as any other machine learning problem, knowl- edge harvesting and ontology learning models exploit the above hypothesis to build feature spaces where instances, i.e., words as in [22] or word pairs as in [27], are represented. These feature spaces are used to determine whether or not new word pairs coming from the text collection have to be included in exist- ing knowledge repositories. Decision models are learnt ∗DISP University Rome “Tor Vergata“ using existing knowledge repositories and then applied to new words or word pairs. Generally, these models use as features all the possible and relevant generalized contexts where words or word pairs can appear. For example, possible features in the word pair classifica- tion problem are ”is a” and ”as well as”. Given the nature of the problem, these feature spaces can then be huge as they include all potential relevant features for a particular relation among words. Relevant features are not known in advance. Yet, large feature spaces can have negative effects on machine learning models such as increasing the …

Content truncated to 500 words