Reranking the Berkeley and Brown Parsers

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 665–668, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Reranking the Berkeley and Brown Parsers∗ Mark Johnson Department of Computing Macquarie University Sydney, Australia mjohnson@science.mq.edu.au Ahmet Engin Ural Cognitive and Linguistic Sciences Brown University Providence, RI, USA aeural@gmail.com Abstract The Brown and the Berkeley parsers are two state-of-the-art generative parsers. Since both parsers produce n-best lists, it is possible to apply reranking techniques to the output of both of these parsers, and to their union. We note that the standard reranker feature set dis- tributed with the Brown parser does not do well with the Berkeley parser, and propose an extended set that does better. An ablation ex- periment shows that different parsers benefit from different reranker features. 1 Introduction Syntactic parsing is the task of identifying the phrases and clauses in natural language sentences. It has been intensively studied primarily because it is generally believed that identifying syntactic struc- ture is a first step towards semantic interpretation. This paper focuses on parsing the Wall Street Jour- nal (WSJ) section of the University of Pennsylva- nia treebank corpus (Marcus et al., 1993). There are a large number of different approaches to this task. For simplicity we focus on two popular gener- ative statistical parsing models: Charniak’s “Maxi- mum Entropy Inspired” parser (Charniak and John- son, 2005) and Petrov’s “split-merge” parser (Petrov et al., 2006). We follow conventional informal usage and refer to these as the “Brown” and the “Berkeley” parsers respectively. ∗ We would like to thank Eugene Charniak and the other members of BLLIP for their helpful advice on this work. Natu- rally all errors remain our own. Briefly, the Berkeley parser is a smoothed PCFG whose non-terminals are refinements of the original treebank grammar obtained by an automatic split- merge procedure, while the Brown parser is effec- tively a smoothed PCFG whose non-terminals en- code a wide variety of manually chosen condition- ing information, such as heads, governors, etc. The Berkeley parser is usually viewed as unlexicalized (although the preterminals may be split so finely that they may be viewed as identifying lexical clus- ters), while essentially every distribution used in the Brown parser conditions on lexical information. Even from this cursory description it is clear that these parsers parsers extract generalizations from the training data in different ways. This paper applies reranking (Collins and Koo, 2005) to the n-best output of both parsers individ- ually, as well as to an n-best list consisting of the union of the outputs of both parsers. We are inter- ested to see whether the same kinds of features im- prove the performance of both the Berkeley and the Brown parsers, or whether successful reranking re- quires features that are specially tuned to the parser it is applied to. Finally, we are interested in the performance of the reranker trained on the union n- best lists. Combining the output of multiple parsers in other …

Content truncated to 500 words