Parse Reranking Based on Higher-Order Lexical Dependencies

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1251–1259, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Abstract Existing work shows that lexical dependen- cies are helpful for constituent tree parsing. However, only first-order lexical dependen- cies have been employed and investigated in previous work. In this paper, we propose a method to employing higher-order1 lexical dependencies for constituent tree evaluation. Our method is based on a parse reranking framework, which provides a constrained search space (via N-best lists or parse forests) and enables our parser to employ relatively complicated dependency features. We eva- luate our models on the Penn Chinese Tree- bank. The highest F1 1 Introduction score reaches 85.74%, thus outperforming all previously reported state-of-the-art systems. The dependency ac- curacy of constituent trees generated by our parser has been significantly improved as well. The most commonly used grammar for constitu- ent structure parsing is probabilistic context-free grammar (PCFG). However, as demonstrated in Klein and Manning (2003a), PCFG estimated straightforwardly from Treebank does not per- form well. The reason is that the basic PCFG has certain recognized drawbacks: its independence assumption is too strong, and it lacks of lexical conditioning (Jurafsky and Martin, 2008). To address these drawbacks, several variants of PCFG-based models have been proposed (Klein and Manning, 2003a; Matsuzaki et al., 2005; Petrov et al., 2006; Petrov and Klein, 2007). Lex- icalized PCFG (LPCFG) (Collins, 1999; Charniak, 2000; Bikel, 2004) is a representative work that tries to ameliorate the deficiency of lexical conditioning. In LPCFG, non-terminals are annotated with lexical heads and the probabil- ities of CFG rules are estimated conditioned upon these lexical heads. Thus LPCFG becomes sensi- tive to lexical heads, and its performance is im- proved. However, the information provided by lexical heads is limited. To obtain higher parsing performance, we must seek additional informa- tion. We believe that dependency trees are good candidates because they encode grammatical re- lations between words and provide much more lexical conditioning than lexical heads for PCFG. Dependency trees are usually factored into sets of lexical dependency parts for evaluation. The order of a lexical dependency part can be defined according to the number of dependency arcs it contains. For example, in Figure 1, dependency is first-order, sibling and grandchild are second- order and grand-sibling and tri-sibling are third- order. During the past few years, higher-order1 lexical dependencies have been successfully used for dependency parsing (McDonald et al., 2005; McDonald and Pereira, 2006; Koo and Collins, 2010). But for constituent tree evaluation, only first-order (bigram) lexical dependencies have been used (Collins, 1996; Klein and Manning, 2003a; Collins and Koo, 2005). However, first- order lexical dependency parts are quite limited and thus lose much of the contextual information within the dependency tree. To improve parsing performance, we propose to evaluate constituent trees with higher-order lexical dependencies. In this paper, we propose a method for evaluat- ing constituent trees using higher-order lexical dependencies within a parse reranking frame- work. We evaluate our method on …

Content truncated to 500 words