Distributional Part-of-Speech Tagging

Distributional Part-of-Speech Tagging Hinrich Schfitze CSLI, Ventura Hall Stanford, CA 94305-4115 , USA emMl: schuetze~csli.stanford.edu URL: ftp://csli.stanford.edu/pub/prosit/DisPosTag.ps Abstract This paper presents an algorithm for tag- ging words whose part-of-speech proper- ties are unknown. Unlike previous work, the algorithm categorizes word tokens in con$ezt instead of word ~ypes. The algo- rithm is evaluated on the Brown Corpus. 1 Introduction Since online text becomes available in ever increas- ing volumes and an ever increasing number of lan- guages, there is a growing need for robust pro- cessing techniques that can analyze text without expensive and time-consuming adaptation to new domains and genres. This need motivates research on fully automatic text processing that may rely on general principles of linguistics and computa- tion, but does not depend on knowledge about individual words. In this paper, we describe an experiment on fully automatic derivation of the knowledge nec- essary for part-of-speech tagging. Part-of-speech tagging is of interest for a number of applications, for example access to text data bases (Kupiec, 1993), robust parsing (Abney, 1991), and general parsing (deMarcken, 1990; Charniak et al., 1994). The goal is to find an unsupervised method for tagging that relies on general distributional prop- erties of text, properties that are invariant across languages and sublanguages. While the proposed algorithm is not successful for all grammatical cat- egories, it does show that fully automatic tagging is possible when demands on accuracy are modest. The following sections discuss related work, de- scribe the learning procedure and evaluate it on the Brown Corpus (Francis and Ku~era, 1982). 2 Related Work The simplest part-of-speech taggers are bigram or trigram models (Church, 1989; Charniak et al., 1993). They require a relatively large tagged training text. Transformation-based tagging as introduced by Brill (1993) also requires a hand- tagged text for training. No pretagged text is nec- essary for Hidden Markov Models (Jelinek, 1985; Cutting et al., 1991; Kupiec, 1992). Still, a lexi- con is needed that specifies the possible parts of speech for every word. Brill and Marcus (1992a) have shown that the effort necessary to construct the part-of-speech lexicon can be considerably re- duced by combining learning procedures and a partial part-of-speech categorization elicited from an informant. The present paper is concerned with tagging languages and sublanguages for which no a priori knowledge about grammatical categories is avail- able, a situation that occurs often in practice (Brill and Marcus, 1992a). Several researchers have worked on learning grammatical properties of words. Elman (1990) trains a connectionist net to predict words, a pro- cess that generates internal representations that reflect grammatical category. Brill et al. (1990) try to infer grammatical category from bi- gram statistics. Finch and Chater (1992) and Finch (1993) use vector models in which words are clustered according to the similarity of their close neighbors in a corpus. Kneser and Ney (1993) present a probabilistic model for entropy maxi- mization that also relies on the immediate neigh- bors of words in a corpus. Biber (1993) ap- plies factor analysis to collocations of …

Content truncated to 500 words