Exploring the Use of Linguistic Features in Domain and Genre Classification

Proceedings of EACL '99 Exploring the Use of Linguistic Features in Domain and Genre Classification Maria Wolters' and Mathias Kirsten 2 1Inst. f. Kommunikationsforschung u. Phonetik, Bonn; wolters@ikp.uni-bonn.de 2German Natl. Res. Center for IT-AiS.KD-, St. Augustin; mathias.kirsten~gmd.de Abstract The central questions are: How useful is information about part-of-speech fre- quency for text categorisation? Is it fea- sible to limit word features to content words for text classifications? This is examined for 5 domain and 4 genre clas- sification tasks using LIMAS, the Ger- man equivalent of the Brown corpus. Be- cause LIMAS is too heterogeneous, nei- ther question can be answered reliably for any of the tasks. However, the re- sults suggest that both questions have to be examined separately for each task at hand, because in some cases, the ad- ditional information can indeed improve performance. 1 Introduction The greater the amounts of text people can ac- cess and have to process, the more important effi- cient methods for text categorisation become. So far, most research has concentrated on content- based categories. But determining the genre of a text can also be very important, for example when having to distinguish an EU press release on the introduction of the euro from a newspaper commentary on the same topic. The results of e.g. (Lewis, 1992; Yang and Ped- ersen, 1997) indicate that for good content clas- sification, we basically need a vector which con- tains the most relevant words of the text. Using n-grams hardly yields significant improvements, because the dimension of the document represen- tation space increases exponentially. But do word- based vectors also work well for genre detection? Or do we need additional linguistically motivated features to capture the different styles of writing associated with different genres? In this paper, we present a pilot study based on a set of easily computable linguistic features, namely the frequency of part-of-speech (POS) tags, and a corpus of German, LIMAS (Glas, 1975), which contains a wide range of different genres. LIMAS is described briefly in Sac. 3, while sections 2 and 4 motivate the choice of features. The text categorisation experiments are described in Sec. 5. 2 Linguistic Cues to Genre 2.1 What is genre? The term "genre" is more frequent in philology and media studies than in mainstream linguistics (Swales, 1990, p.38). When it is not used synony- mously with the terms "register" or "style", genre is defined on the basis of non-linguistic criteria. For example, (Biber, 1988) characterises genres in terms of author/speaker purpose, while text types classify texts on the basis of text-internal criteria. Swales phrases this more precisely: Genres are collections of communicative events with shared communicative purposes which can vary in their prototypicality. These communicative purposes are determined by the discourse community which produces and reads texts belonging to a genre. But how can we extract its communicative pur- pose from a given text? First of all, we need to define the genres we want to detect. The defi- nitions which were used in this study …

Content truncated to 500 words