Utilizing Extra-Sentential Context for Parsing

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 23–33, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Utilizing Extra-sentential Context for Parsing Jackie Chi Kit Cheung and Gerald Penn Department of Computer Science University of Toronto Toronto, ON, M5S 3G4, Canada {jcheung,gpenn}@cs.toronto.edu Abstract Syntactic consistency is the preference to reuse a syntactic construction shortly after its appearance in a discourse. We present an anal- ysis of the WSJ portion of the Penn Tree- bank, and show that syntactic consistency is pervasive across productions with various left- hand side nonterminals. Then, we implement a reranking constituent parser that makes use of extra-sentential context in its feature set. Using a linear-chain conditional random field, we improve parsing accuracy over the gen- erative baseline parser on the Penn Treebank WSJ corpus, rivalling a similar model that does not make use of context. We show that the context-aware and the context-ignorant rerankers perform well on different subsets of the evaluation data, suggesting a combined ap- proach would provide further improvement. We also compare parses made by models, and suggest that context can be useful for parsing by capturing structural dependencies between sentences as opposed to lexically governed de- pendencies. 1 Introduction Recent corpus linguistics work has produced ev- idence of syntactic consistency, the preference to reuse a syntactic construction shortly after its ap- pearance in a discourse (Gries, 2005; Dubey et al., 2005; Reitter, 2008). In addition, experimental stud- ies have confirmed the existence of syntactic prim- ing, the psycholinguistic phenomenon of syntactic consistency1. Both types of studies, however, have 1Whether or not corpus-based studies of consistency have any bearing on syntactic priming as a reality in the human mind limited the constructions that are examined to partic- ular syntactic constructions and alternations. For in- stance, Bock (1986) and Gries (2005) examine spe- cific constructions such as the passive voice, dative alternation and particle placement in phrasal verbs, and Dubey et al. (2005) deal with the internal struc- ture of noun phrases. In this work, we extend these results and present an analysis of the distribution of all syntactic productions in the Penn Treebank WSJ corpus. We provide evidence that syntactic consis- tency is a widespread phenomenon across produc- tions of various types of LHS nonterminals, includ- ing all of the commonly occurring ones. Despite this growing evidence that the probability of syntactic constructions is not independent of the extra-sentential context, current high-performance statistical parsers (e.g. (Petrov and Klein, 2007; Mc- Closky et al., 2006; Finkel et al., 2008)) rely solely on intra-sentential features, considering the partic- ular grammatical constructions and lexical items within the sentence being parsed. We address this by implementing a reranking parser which takes ad- vantage of features based on the context surrounding the sentence. The reranker outperforms the genera- tive baseline parser, and rivals a similar model that does not make use of context. We show that the context-aware and the context-ignorant models per- form well on different subsets of the …

Content truncated to 500 words