Reranking a wide-coverage ccg parser

Reranking a wide-coverage CCG parser Dominick Ng and Matthew Honnibal and James R. Curran School of Information Technologies University of Sydney NSW 2006, Australia {dong7223,mhonn,james}@it.usyd.edu.au Abstract n-best parse reranking is an important technique for improving the accuracy of statistical parsers. Reranking is not con- strained by the dynamic programming re- quired for tractable parsing, so arbitrary features of each parse may be considered. We adapt the reranking features and methodology used by Charniak and John- son (2005) for the C&C Combinatory Cat- egorial Grammar parser, and develop new features based on the richer formalism. The reranker achieves a labeled depen- dency F-score of 87.59%, which is a sig- nificant improvement over prior results. 1 Introduction Accurate syntactic parsing has proven to be criti- cal for many tasks in natural language processing (NLP), including semantic role labeling (Gildea and Jurafsky, 2002), question answering (Echihabi and Marcu, 2003), and machine translation (De- Neefe and Knight, 2009). Improved parser accu- racy benefits many downstream tasks in the field. One method of improving parsing accuracy is reranking – the process of reordering the top n analyses as determined by a base parser (Collins, 2000). The statistical models used in phrase- structure and dependency parsers rely on dynamic programming algorithms that restrict possible fea- tures to a local context. This is necessary for effi- cient decoding of the potential parse forest, ensur- ing tractability at the cost of excluding any non- local features from consideration. Reranking op- erates over complete trees that are the most prob- able derivations under the dynamic programming model, allowing arbitrary complex features of the parse to be incorporated without sacrificing effi- ciency. Poor local decisions made by parsers are easier to model and capture in the reranking phase. Collins (2000) reports a 1.55% accuracy im- provement with reranking for the Collins parser, and Charniak and Johnson (2005) reports a 1.3% improvement for a reranked Charniak parser. An open question is how well reranking applies to parsers of different design to the Charniak and Collins parsers. An attempt to port the Char- niak and Johnson reranker for the Berkeley parser (Petrov et al., 2006) produced only minimal ac- curacy improvements (Johnson and Ural, 2010), suggesting that careful feature engineering is nec- essary for good performance. In this paper we describe the implementation of a discriminative maximum entropy reranker for the C&C parser (Clark and Curran, 2007), a state- of-the-art system based on Combinatory Catego- rial Grammar (CCG). We reimplement the features described in Charniak and Johnson (2005) to suit the CCG parser and replicate the Charniak reranker setup. Our experiments show that the PCFG- style features are less effective at reranking CCG than Penn Treebank-style trees. We hypothesise that the binary-branching structure of CCG is the cause, as CCG trees are deeper and create differ- ent constituent structures compared to Penn Tree- bank trees. To address this, we develop a number of new features to take advantage of the more de- tailed formalism and the evaluation over recovered dependencies. We also …

Content truncated to 500 words