A Statistical Decision Making Method: A Case Study on Prepositional Phrase Attachment

A Statistical Decision Making Method: A Case Study on Prepositional Phrase Attachment* Mehmet Kayaalp, Ted Pedersen and Rebecca Bruce Department of Computer Science & Engineering Southern Methodist University Dallas, TX 75275-0122 {kayaalp, pedersen, rbruce}@seas, smu. edu Abstract Statistical classification methods usually rely on a single best model to make ac- curate predictions. Such a model aims to maximize accuracy by balancing precision and recall. The Model Switching method as presented in this paper performs with higher predictive accuracy and 100% recall by using a set of decomposable models in- stead of a single one. The implemented system, MS1, is tested on a case study, predicting Prepositional Phrase Attachment (PPA). The results show that iV is more accurate than other statistical techniques that select single models for classification and competitive with other successful NLP approaches in PPA disambiguation. The Model Switch- ing method may be preferable to other methods because of its generality (i.e., wide range of applicability), and its competitive accuracy in prediction. It may also be used as an analytical tool to investigate the na- ture of the domain and the characteristics of the data with the help of generated mod- els. 1 Introduction Decision problems are classically defined as problems whose answers fall in either of two classes: Yes and No (Garey and Johnson, 1979). Optimization prob- lems are another class of problems that maximize or minimize some value; however, they can be cast as decision problems as well (Cormen et al., 1990). Classification problems incorporate the characteris- tics of both: A classification problem is a decision *This research was supported in part by the Office of Naval Research under grant number N00014-95-1-0776 problem, in which a decision is made (a class is se- lected) that maximizes a utility function (yon Neu- mann and Morgenstern, 1953). The Model Switch- ing method as proposed in this paper can be used with any utility function (decision criterion) for any decision problem with categorical data that can be represented as a tuple (C, F1, F2, ..., Fn) of a class variable C and some feature variables F{1 .... }. In the following sections, we will describe the Prepositional Phrase Attachment (PPA) problem and various approaches to solving it. After dis- cussing the statistical concepts used in this work, we will introduce the concept of Model Switching, why it is needed, how it works, and our experience on the PPA problem with Model Switching. Comparisons with earlier works on corpus-based PPA prediction and conclusions will follow. 2 PPA Problem Resolving the PPA problem is a common problem in any NLP system that deals with syntactic parsing or text understanding. The Naive Bayes classifier and leading machine learning systems, such as C4.5 (Quinlan, 1993), CN2 (Clark and Niblett, 1989) and PEBLS (Cost and Sahberg, 1993), fail to provide pre- diction with competitive accuracy rates on this prob- lem (see Table 4 on page 40). A sentence can be so ambiguous that it may not be possible to determine the correct attachment without extra …

Content truncated to 500 words