Improved Parsing and POS Tagging Using Inter-Sentence Consistency Constraints

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1434–1444, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Improved Parsing and POS Tagging Using Inter-Sentence Consistency Constraints Alexander M. Rush1∗ Roi Reichart1∗ Michael Collins2 Amir Globerson3 1MIT CSAIL, Cambridge, MA, 02139, USA {srush|roiri}@csail.mit.edu 2Department of Computer Science, Columbia University, New-York, NY 10027, USA mcollins@cs.columbia.edu 3School of Computer Science and Engineering, The Hebrew University, Jerusalem, 91904, Israel gamir@cs.huji.ac.il Abstract State-of-the-art statistical parsers and POS taggers perform very well when trained with large amounts of in-domain data. When train- ing data is out-of-domain or limited, accuracy degrades. In this paper, we aim to compen- sate for the lack of available training data by exploiting similarities between test set sen- tences. We show how to augment sentence- level models for parsing and POS tagging with inter-sentence consistency constraints. To deal with the resulting global objective, we present an efficient and exact dual decomposition de- coding algorithm. In experiments, we add consistency constraints to the MST parser and the Stanford part-of-speech tagger and demonstrate significant error reduction in the domain adaptation and the lightly supervised settings across five languages. 1 Introduction State-of-the-art statistical parsers and POS taggers perform very well when trained with large amounts of data from their test domain. When training data is out-of-domain or limited, the performance of the re- sulting model often degrades. In this paper, we aim to compensate for the lack of available training data by exploiting similarities between test set sentences. Most parsing and tagging models are defined at the sentence-level, which makes such inter-sentence in- formation sharing difficult. We show how to aug- ment sentence-level models with inter-sentence con- straints to encourage consistent descisions in similar ∗ Both authors contributed equally to this work. contexts, and we give an efficient algorithm with for- mal guarantees for decoding such models. In POS tagging, most taggers perform very well on word types that they have observed in training data, but they perform poorly on unknown words. With a global objective, we can include constraints that encourage a consistent tag across all occur- rences of an unknown word type to improve accu- racy. In dependency parsing, the parser can benefit from surface-level features of the sentence, but with sparse or out-of-domain training data these features are very noisy. Using a global objective, we can add constraints that encourage similar surface-level con- texts to exhibit similar syntactic behaviour. The first contribution of this work is the use of Markov random fields (MRFs) to model global con- straints between sentences in dependency parsing and POS tagging. We represent each word as a node, the tagging or parse decision as its label, and add constraints through edges. MRFs allow us to include global constraints tailored to these problems, and to reason about inference in the corresponding global models. The second contribution is an efficient dual de- composition algorithm for decoding a global ob- jective with inter-sentence constraints. These con- …

Content truncated to 500 words