Prepositional Phrase Attachment through a Backed-off Model

Prepositional Phrase Attachment through a Backed-Off Model Michael Collins and James Brooks Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 {mcollins, jbrooks} @gradient.cis.upenn.edu Abstract Recent work has considered corpus-based or statistical approaches to the problem of prepositional phrase attachment ambiguity. Typically, ambiguous verb phrases of the form v rip1 p rip2 are resolved through a model which considers values of the four head words (v, nl, p and 77,2). This paper shows that the problem is analogous to n-gram language models in speech recognition, and that one of the most common methods for language modeling, the backed-off estimate, is applicable. Results on Wall Street Journal data of 84.5% accuracy are obtained using this method. A surprising result is the importance of low-count events - ignoring events which occur less than 5 times in training data reduces performance to 81.6%. 1 Introduction Prepositional phrase attachment is a common cause of structural ambiguity in natural language. For example take the following sentence: Pierre Vinken, 61 years old, joined the board as a nonexecutive director. The PP 'as a nonexecutive director' can either attach to the NP 'the board' or to the VP 'joined', giving two alternative structures. (In this case the VP attachment is correct): NP-attach: (joined ((the board) (as a nonexecutive director))) VP-attach: ((joined (the board)) (as a nonexecutive director)) Work by Ratnaparkhi, Reynar and Roukos [RRR94] and Brill and Resnik [BR94] has considered corpus-based approaches to this problem, using a set of examples to train a model which is then used to make attachment decisions on test data. Both papers describe methods which look at the four head words involved in the attachment - the VP head, the first NP head, the preposition and the second NP head (in this case joined, board, as and director respectively). This paper proposes a new statistical method for PP-attachment disambiguation based on the four head words. 2"7 2 Background 2.1 Training and Test Data The training and test data were supplied by IBM, being identical to that used in [RRR94]. Examples of verb phrases eontMning a (v np pp) sequence had been taken fl'om the Wall Street Journal Treebank [MSM93]. For each such VP the head verb, first head noun, preposition and second head noun were extracted, along with the attachment decision (1 for noun attachment, 0 for verb). For example the verb phrase: ((joined (the board)) (as a nonexecutive director)) would give the quintuple: 0 joined board as director The elements of this quintuple will from here on be referred to as the random variables A, V, N1, P, and N2. In the above verb phrase A = 0, V = joined, N1 = board, P = as, and N2 = director. The data consisted of training and test files of 20801 and 3097 quintuples respectively. In addition, a development set of 4039 quintuples was also supplied. This set was used during development of the attachment algorithm, ensuring that there was no implicit training of the method on …

Content truncated to 500 words