Part-of-Speech Tagging With Neural Networks

PART-OF-SPEECH TAGGING WITH NEURAL NETWORKS Hehnut Schmid Institute for Computational Linguistics, Azenbergstr.12, 70174 Stuttgart, Germany, schmid@ims.uni-stuttgart.de Topic area: large text corpora, part-of-speech tag- ging, neural networks 1 ABSTRACT Text corpora which are tagged with part-of-speech in- formation are useful in many areas of linguistic re- search. In this paper, a new part-of-speech tagging method hased on neural networks (Net-Tagger) is pre- sented and its performance is compared to that of a llMM-tagger (Cutting et al., 1992) and a trigram- based tagger (Kempe, 1993). It is shown that the Net-Tagger performs as well as the trigram-based tag- ger and better than the iIMM-tagger. 2 INTRODUCTION Words are often ambiguous in their part of speech. The English word store for example can be either a noun, a finite verb or an infinitive. In an utterance, this ambiguity is normally resolved by the context of a word: e.g. in the seutence "The 1977 P6's could store two pages of data. ", store can only be an intluitive. A part-of-speech tagger is a system which automat- ically assigns the part of speech to words using con- textual information. Potential applications for part- of-speech taggers exist in many areas inclnding speech recognition, speech synthesis, machine translation and information retrieval. l)ifi'ereut methods have been used for the im plemen- ration of part-of-speech taggers. TAGGIT (Greene, Rnbin, 1971), an early system, which was used for the initial tagging of the Brown corpus was rule-based. It was able to assign the correct part-of-speech to about 77 % of the words in the Brown corpus. In another approach contextual dependencies are modelled statistically. Churcb (1988) and Kempe (1993) use second order Markov Models and train their systems on large handtagged corpora. Using this metbod, they are able to tag more than 96 % of their test words with the correct part-of-speech. The need for reliably tagged training data, however, is a prob- lem for languages, where such data is not available in sufficient quantities. Jelinek (1985) and Cutting et al. (1992) circumvent this problem by training their taggers on untagged data using tile Itaum-Welch algo- rithm (also know as the forward-backward algorithm). They report rates of correctly tagged words which are comparable to that presented by Church (1988) and Kempe (1993). A third and rather new approach is tagging with artificial neural networks. In the area of speech recog- nition neural networks have been used for a decade r, ow. They have shown performances comparable to that of IIidden Ivlarkov model systems or even better (Lippmann, 1989). Part-of-speech prediction is an- other area, closer to POS tagging, where neural net- works have been applied successfidly. Nakamura el; al. (1990) trained a d-layer feed-forward network with up to three preceding part-of-speech tags ,as input to predict the word category of the next word. The pre- diction accuracy was similar to that of a trigram-b,~sed predictor. Using tile predictor, Nakamura et al. were able to improve the recognition rate of their speech recognition system from 81.0 % to 86.9 %. Federici …

Content truncated to 500 words