Experiments with Sentence Classification

Proceedings of the 2006 Australasian Language Technology Workshop (ALTW2006), pages 18–25. Experiments with Sentence Classification Anthony Khoo, Yuval Marom and David Albrecht Faculty of Information Technology, Monash University Clayton, VICTORIA 3800, AUSTRALIA nthony ak@yahoo.com, {yuvalm,dwa}@csse.monash.edu.au Abstract We present a set of experiments involving sentence classification, addressing issues of representation and feature selection, and we compare our findings with simi- lar results from work on the more general text classification task. The domain of our investigation is an email-based help-desk corpus. Our investigations compare the use of various popular classification algo- rithms with various popular feature selec- tion methods. The results highlight simi- larities between sentence and text classifi- cation, such as the superiority of Support Vector Machines, as well as differences, such as a lesser extent of the usefulness of features selection on sentence classifi- cation, and a detrimental effect of com- mon preprocessing techniques (stop-word removal and lemmatization). 1 Introduction Classification tasks applied to textual data have been receiving increasing attention due to the ex- plosion in digital presentation and storage of tex- tual information, such as web pages, emails, publi- cations, and discussion forums. The bulk of the re- search concerns the classification of complete doc- uments, such as spam detection in emails (Drucker et al., 1999), and the classification of news arti- cles (Yang and Pedersen, 1997; Joachims, 1998). These kinds of tasks are widely known as text clas- sification (TC). A text document is best character- ized by the words and terms it contains, and conse- quently the representation of textual data is often of a very high dimensionality. Thus, an important aspect of TC is feature selection (Yang and Peder- sen, 1997; Forman, 2003). There are numerous examples of textual docu- ments whose content conveys communication be- tween multiple parties. In such documents, it may be useful to classify individual sentences that ex- press communicative acts, either to obtain a more meaningful description of the documents, or sim- ply to extract meaningful components, such as ac- tion items or opinions. The computational lin- guistics community devotes considerable research into speech and dialogue acts, and has developed a markup convention for coding both spoken and written language (Core and Allen, 1997, for exam- ple). The classifications we use for sentences are inspired by such conventions. Although there are existing implementations of sentence classification (SC) (Zhou et al., 2004; Wang et al., 2005; McKnight and Srinivasan, 2003), including ones where sentences convey communicative acts (Cohen et al., 2004; Corston- Oliver et al., 2004; Ivanovic, 2005), comparatively little attention has been given to SC in general. In particular, there are no empirical demonstrations of the effect of feature selection in SC tasks, to the best of our knowledge. This paper presents a study into sentence clas- sification, with particular emphasis on representa- tional issues of extracting features from sentences, and applying feature selection (FS) methods. We experiment with various widely accepted FS meth- ods and classification algorithms, and relate our findings to results from TC reported in the …

Content truncated to 500 words