Compacting the Penn Treebank Grammar

Compacting the Penn Treebank Grammar Alexander Krotov and Mark Hepple and Robert Gaizauskas and Yorick Wilks Department of Computer Science, Sheffield University 211 Portobello Street, Sheffield S1 4DP, UK {alexk, hepple, robertg, yorick}@dcs.shef.ac.uk Abstract Treebanks, such as the Penn Treebank (PTB), offer a simple approach to obtaining a broad coverage grammar: one can simply read the grammar off the parse trees in the treebank. While such a grammar is easy to obtain, a square-root rate of growth of the rule set with corpus size suggests that the derived grammar is far from complete and that much more tree- banked text would be required to obtain a com- plete grammar, if one exists at some limit. However, we offer an alternative explanation in terms of the underspecification of structures within the treebank. This hypothesis is ex- plored by applying an algorithm to compact the derived grammar by eliminating redund- ant rules - rules whose right hand sides can be parsed by other rules. The size of the result- ing compacted grammar, which is significantly less than that of the full treebank grammar, is shown to approach a limit. However, such a compacted grammar does not yield very good performance figures. A version of the compac- tion algorithm taking rule probabilities into ac- count is proposed, which is argued to be more linguistically motivated. Combined with simple thresholding, this method can be used to give a 58% reduction in grammar size without signi- ficant change in parsing performance, and can produce a 69% reduction with some gain in re- call, but a loss in precision. 1 Introduction The Penn Treebank (PTB) (Marcus et al., 1994) has been used for a rather simple approach to deriving large grammars automatically: one where the grammar rules are simply 'read off' the parse trees in the corpus, with each local subtree providing the left and right hand sides of a rule. Charniak (Charniak, 1996) reports precision and recall figures of around 80% for a parser employing such a grammar. In this paper we show that the huge size of such a tree- bank grammar (see below) can be reduced in size without appreciable loss in performance, and, in fact, an improvement in recall can be achieved. Our approach can be generalised in terms of Data-Oriented Parsing (DOP) methods (see (Bonnema et al., 1997)) with the tree depth of 1. However, the number of trees produced with a general DOP method is so large that Bonnema (Bonnema et al., 1997) has to resort to restrict- ing the tree depth, using a very domain-specific corpus such as ATIS or OVIS, and parsing very short sentences of average length 4.74 words. Our compaction algorithm can be easily exten- ded for the use within the DOP framework but, because of the huge size of the derived grammar (see below), we chose to use the simplest PCFG framework for our experiments. We are concerned with the nature of the rule set extracted, and how it can be improved, with regard both …

Content truncated to 500 words