Semi-supervised dependency parsing using generalized tri-training

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1065–1073, Beijing, August 2010 Semi-supervised dependency parsing using generalized tri-training Anders Søgaard and Christian Rishøj Center for Language Technology University of Copenhagen {soegaard|crjensen}@hum.ku.dk Abstract Martins et al. (2008) presented what to the best of our knowledge still ranks as the best overall result on the CONLL- X Shared Task datasets. The paper shows how triads of stacked dependency parsers described in Martins et al. (2008) can label unlabeled data for each other in a way similar to co-training and produce end parsers that are significantly better than any of the stacked input parsers. We evaluate our system on five datasets from the CONLL-X Shared Task and ob- tain 10–20% error reductions, incl. the best reported results on four of them. We compare our approach to other semi- supervised learning algorithms. 1 Introduction Semi-supervised learning of structured variables is a difficult problem that has received consid- erable attention recently, but most results have been negative (Abney, 2008). This paper uses stacked learning (Wolpert, 1992) to reduce struc- tured variables, i.e. dependency graphs, to multi- nomial variables, i.e. attachment and labeling decisions, which are easier to manage in semi- supervised learning scenarios, and which can later be combined into dependency trees using parsing algorithms for arc-factored dependency parsing. Our approach thus combines ensemble- based methods and semi-supervised learning. Ensemble-based methods such as stacked learning are used to reduce the instability of clas- sifiers, to average out their errors and to com- bine the strengths of diverse learning algorithms. Ensemble-based methods have attracted a lot of attention in dependency parsing recently (Sagae and Lavie, 2006; Hall et al., 2007; Nivre and McDonald, 2008; Martins et al., 2008; Fishel and Nivre, 2009; Surdeanu and Manning, 2010). Nivre and McDonald (2008) were first to intro- duce stacking in the context of dependency pars- ing. Semi-supervised learning is typically moti- vated by data sparseness. For many classifi- cation tasks in natural language processing, la- beled data can be in short supply but unla- beled data is more readily available. Semi- supervised methods exploit unlabeled data in ad- dition to labeled data to improve performance on classification tasks. If the predictions of a learner l on unlabeled data are used to improve a learner l′ in semi-supervised learning, the ro- bustness of learning will depend on the stabil- ity of l. Combining ensemble-based and semi- supervised methods may thus lead to more ro- bust semi-supervised learning. Ensemble-based and semi-supervised meth- ods are some of the areas that receive most at- tention in machine learning today, but relatively little attention has been given to combining these methods (Zhou, 2009). Semi-supervised learn- ing algorithms can be categorized with respect to the number of views, i.e. the number of fea- ture sets, and the number of learners used to in- form each other (Hady and Schwenker, 2008). Self-training and expectation maximization are perhaps the best known semi-supervised learn- ing algorithms (Abney, 2008). They are both single-view and single-learner …

Content truncated to 500 words