Improving Text Classification by a Sense Spectrum Approach to Term Expansion

Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), pages 183–191, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Improving Text Classification by a Sense Spectrum Approach to Term Expansion Peter Wittek Department of Computer Science National University of Singapore Computing 1, Law Link Singapore 117590 wittek@comp.nus.edu.sg S´andor Dar´anyi Swedish School of Library and Information Science G¨oteborg University & University of Bor˚as All´egatan 1 50190 Bor˚as, Sweden sandor.daranyi@hb.se Chew Lim Tan Department of Computer Science National University of Singapore Computing 1, Law Link Singapore 117590 tancl@comp.nus.edu.sg Abstract Experimenting with different mathematical objects for text representation is an important step of building text classification models. In order to be efficient, such objects of a for- mal model, like vectors, have to reasonably re- produce language-related phenomena such as word meaning inherent in index terms. We in- troduce an algorithm for sense-based seman- tic ordering of index terms which approxi- mates Cruse’s description of a sense spectrum. Following semantic ordering, text classifica- tion by support vector machines can benefit from semantic smoothing kernels that regard semantic relations among index terms while computing document similarity. Adding ex- pansion terms to the vector representation can also improve effectiveness. This paper pro- poses a new kernel which discounts less im- portant expansion terms based on lexical re- latedness. 1 Introduction Generally, building an automated text classification system consists of two key subtasks. The first task is text representation which converts the content of documents into compact format so that they can be further processed by the text classifiers. Another task is to learn the model of a text classifier which is used to classify the unlabeled documents. This paper proposes a substantially new model for text representation to improve effectiveness of text clas- sification by semantic ordering. Our motivation for the research presented here came from (Dorrer et al., 2001) who demonstrated the viability of database searching by visible light using a quantum algorithm, albeit on meaningless items. The question was, what kind of document representation would be necessary to extend their in-principle results to include semantics, one that has been leading us to test both periodic and non- periodic functions for this purpose. Since represen- tation and retrieval by colors was implied in their method, we speculated that the following compo- nents could be useful in a rephrased model: (a) a metaphorically presented spectral expression of lexical semantic phenomena, (b) a ranked one- dimensional condensate of multidimensional sense structure, and (c) representation of documents and queries by functions in L2 space with a similarity measure. Our anticipation was that by matching these components, a new model could demonstrate new capacities in general, and contribute to comput- ing meaning by waves in particular. Semantic ordering (component b) is an approxi- mation of what (Cruse, 1986) referred to as a sense spectrum, i.e. a series of points - called local senses and constituting lexical units -, in a one-dimensional semantic continuum (component a). Apart from dif- ferentiating between the conceptual content of the …

Content truncated to 500 words