Pipeline Iteration

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 952–959, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Pipeline Iteration Kristy Hollingshead and Brian Roark Center for Spoken Language Understanding, OGI School of Science & Engineering Oregon Health & Science University, Beaverton, Oregon, 97006 USA {hollingk,roark}@cslu.ogi.edu Abstract This paper presents pipeline iteration, an ap- proach that uses output from later stages of a pipeline to constrain earlier stages of the same pipeline. We demonstrate sig- nificant improvements in a state-of-the-art PCFG parsing pipeline using base-phrase constraints, derived either from later stages of the parsing pipeline or from a finite- state shallow parser. The best performance is achieved by reranking the union of un- constrained parses and relatively heavily- constrained parses. 1 Introduction A “pipeline” system consists of a sequence of pro- cessing stages such that the output from one stage provides the input to the next. Each stage in such a pipeline identifies a subset of the possible solutions, and later stages are constrained to find solutions within that subset. For example, a part-of-speech tagger could constrain a “base phrase” chunker (Rat- naparkhi, 1999), or the n-best output of a parser could constrain a reranker (Charniak and Johnson, 2005). A pipeline is typically used to reduce search complexity for rich models used in later stages, usu- ally at the risk that the best solutions may be pruned in early stages. Pipeline systems are ubiquitous in natural lan- guage processing, used not only in parsing (Rat- naparkhi, 1999; Charniak, 2000), but also machine translation (Och and Ney, 2003) and speech recogni- tion (Fiscus, 1997; Goel et al., 2000), among others. Despite the widespread use of pipelines, they have been understudied, with very little work on gen- eral techniques for designing and improving pipeline systems (although cf. Finkel et al. (2006)). This pa- per presents one such general technique, here ap- plied to stochastic parsing, whereby output from later stages of a pipeline is used to constrain earlier stages of the same pipeline. To our knowledge, this is the first time such a pipeline architecture has been proposed. It may seem surprising that later stages of a pipeline, already constrained to be consistent with the output of earlier stages, can profitably inform the earlier stages in a second pass. However, the richer models used in later stages of a pipeline pro- vide a better distribution over the subset of possible solutions produced by the early stages, effectively resolving some of the ambiguities that account for much of the original variation. If an earlier stage is then constrained in a second pass not to vary with re- spect to these resolved ambiguities, it will be forced to find other variations, which may include better so- lutions than were originally provided. To give a rough illustration, consider the Venn di- agram in Fig. 1(i). Set A represents the original sub- set of possible solutions passed along by the earlier stage, and the dark shaded region represents high- probability …

Content truncated to 500 words