Semi-supervised Dependency Parsing using Bilexical Contextual Features from Auto-Parsed Data

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1348–1353, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Semi-supervised Dependency Parsing using Bilexical Contextual Features from Auto-Parsed Data Eliyahu Kiperwasser Computer Science Department Bar-Ilan University Ramat-Gan, Israel elikip@gmail.com Yoav Goldberg Computer Science Department Bar-Ilan University Ramat-Gan, Israel yoav.goldberg@gmail.com Abstract We present a semi-supervised approach to improve dependency parsing accuracy by using bilexical statistics derived from auto-parsed data. The method is based on estimating the attachment potential of head-modifier words, by taking into ac- count not only the head and modifier words themselves, but also the words sur- rounding the head and the modifier. When integrating the learned statistics as features in a graph-based parsing model, we ob- serve nice improvements in accuracy when parsing various English datasets. 1 Introduction We are concerned with semi-supervised depen- dency parsing, namely how to leverage large amounts of unannotated data, in addition to anno- tated Treebank data, to improve dependency pars- ing accuracy. Our method (Section 2) is based on parsing large amounts of unannotated text us- ing a baseline parser, extracting word-interaction statistics from the automatically parsed corpus, and using these statistics as the basis of additional parser features. The automatically-parsed data is used to acquire statistics about lexical interactions, which are too sparse to estimate well from any realistically-sized Treebank. Specifically, we at- tempt to infer a function assoc(head, modifer) measuring the “goodness” of head-modifier rela- tions (“how good is an arc in which black is a mod- ifier of jump”). A similar approach was taken by Chen et al. (2009) and Van Noord et al. (2007). We depart from their work by extending the scoring to include a wider lexical context. That is, given the sentence fragment in Figure 1, we score the (incorrect) dependency arc (black, jump) based on the triplets (the black fox, will jump over). Learn- ing a function between word triplets raises an extreme data sparsity issue, which we deal with by decomposing the interaction between triplets to a sum of interactions between word pairs. The decomposition we use is inspired by recent work in word-embeddings and dense vector rep- resentations (Mikolov et al., 2013a; Mnih and Kavukcuoglu, 2013). Indeed, we initially hoped to leverage the generalization abilities associated with vector-based representations. However, we find that in our setup, reverting to direct count- based statistics achieve roughly the same results (Section 3). Our derived features improve the accuracy of a first-order dependency parser by 0.75 UAS points (absolute) when evaluated on the in-domain WSJ test-set, obtaining a final accuracy of 92.32 UAS for a first-order parser. When comparing to the strong baseline of using Brown-clusters based fea- tures (Koo et al., 2008), we find that our triplets- based method outperform them by over 0.27 UAS points. This is in contrast to previous works (e.g. (Bansal et al., 2014)) in which improvements over using Brown-clusters features were achieved only by adding to the cluster-based features, not by re- placing them. As expected, …

Content truncated to 500 words