Pattern-Based Disambiguation for Natural Language Processing

Eric Brill Microsoft Research One Microsoft Way Redmond, Wa. 98052 brill@microsoft.com 'C 1 Abstract A wide range of natural language problems can be viewed as disambiguating between a small set of alternatives based upon the string context surrounding the ambiguity site. In this paper we demonstrate that classification accuracy can be improved by invoking a more descriptive feature set than what is typically used. We present a technique that disambiguates by learning regular expressions describing the stnng contexts in which the ambiguity sites appear. Introduction Many natural language tasks are essentially n- way classification problems, where classification decisions are made from a small set of choices, based upon the linguistic context in which the ambiguity site occurs. Examples of such tasks include: confusable word set disambiguation; word sense disambiguation; determining such lexical features as pronoun case and determiner number for machine translation; part of speech tagging; named entity labeling; spelling correction; and some formulations of skeletal parsing. Very similar feature sets have been used across machine learning algorithms and across classification problems. For example, in confusable word set disambiguation, systems typically use as features the occurrence of a particular word within a window of +/- n words of the target, and collocations based on the words and part of speech tags of up to two words to the left and two words to the fight of the target. Below we present a machine learning algorithm that learns from a much richer feature set than that typically used for classification in natural language. Our algorithm learns rule sequences for n-way classification, where the condition of a rule can be a restricted regular expression on the string context in which the ambiguity site appears. We demonstrate that using this more powerful feature space leads to an improvement in disambiguation performance on confusable words. 1 Previous Work: Richer Features Most previous work applying machine learning to linguistic disambiguafion has used as features very local collocational information as well as the presence of a word within a fixed window of an ambiguity site. Indeed, one of the great insights in both speech recognition and natural language processing is the realization that fixed local cues provide a great deal of useful information. While the n-gram reins supreme in language modeling, there has been some interesting work done building language models based on linguistically richer features. Bahl, Brown et al. (1989) describe a language model that builds a decision tree that is allowed to ask questions about the history up to twenty words back. Saul and Pereira (1997) describe a language model that can in essence skip over uninformative words in the history. Della Pietra et al. (1994) discuss an approach to language modeling based on link grammars, where the model can look beyond the two previous words to condition on linguistically relevant words in the history. The language model described by Chelba and Jelinek (1998) similarly conditions on linguistically relevant words by assigning partial phrase structure to the history and percolating headwords. Samuellson, Tapanainen et …

Content truncated to 500 words