Feature Weighting for Co-occurrence-based Classification of Words

Feature Weighting for Co-occurrence-based Classification of Words Viktor PEKAR CLG, U. of Wolverhampton Wolverhampton UK, WV1 1SB v.pekar@wlv.ac.uk Michael KRKOSKA Mentasys GmbH Schonfeldstrasse 8 Karlsruhe, Germany, 76131 michael@mentasys.de Steffen STAAB Ontoprise GmbH & Institute AIFB, U. of Karlsruhe Karlsruhe, Germany, 76128 staab@aifb.uni-karlsruhe.de Abstract* The paper comparatively studies methods of feature weighting in application to the task of cooccurrence-based classification of words according to their meaning. We explore parameter optimization of several weighting methods frequently used for similar problems such as text classification. We find that successful application of all the methods crucially depends on a number of parameters; only a carefully chosen weighting procedure allows to obtain consistent improvement on a classifier learned from non-weighted data. 1 Introduction Lexical repositories like thesauri and lexicons are today a key component of many NLP technologies, where they serve as background knowledge for processing the semantics of text. But, as is well known, manual compilation of such resources is a very costly procedure, and their automated construction is an important research issue. One promising possibility to speed up the lexical acquisition process is to glean the semantics of words from a corpus by adopting the co- occurrence model of word meaning. Previous research has investigated a wide range of its applications, including automatic construction of thesauri, their enrichment, acquisition of bilingual lexicons, learning of information extraction patterns, named entity classification and others.. The basic idea behind the approach is that the distribution of a word across lexical contexts (other words and phrases it co-occurs with) is highly indicative of its meaning. The method represents the meaning of a word as a vector where each feature corresponds to a context and its value to the frequency of the word’s occurring in that context. Once such representation is built, machine learning techniques can be used to perform various lexical acquisition tasks, e.g. automatically classify or cluster words according to their meaning. However, using natural language words as features inevitably results in very noisy .* The study was partially supported by the Russian Foundation Basic Research grant #03-06-80008. representations. Because of their inherent polysemy and synonymy, many context words become ambiguous or redundant features. It is therefore desirable to determine a measure of usefulness of each feature and weight it accordingly. Still, despite a wide variety of feature weighting methods existing in machine learning, these methods are poorly explored in application to lexical acquisition. There have been a few studies (e.g., Lin, 1998; Ciaramita, 2002; Alfonseca and Manandhar, 2002) where word representations are modified through this or that kind of feature weighting. But in these studies it is performed only as a standard pre-processing step on the analogy with similar tasks like text categorization, and the choice of a particular weighting procedure is seldom motivated. To our knowledge, there is no work yet on evaluation and comparison of different weighting methods for lexical acquisition. The goal of this paper is to comparatively study a number of popular feature weighting methods in application to the task of …

Content truncated to 500 words