An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 551–560, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing Jun Suzuki, Hideki Isozaki NTT CS Lab., NTT Corp. Kyoto, 619-0237, Japan jun@cslab.kecl.ntt.co.jp isozaki@cslab.kecl.ntt.co.jp Xavier Carreras, and Michael Collins MIT CSAIL Cambridge, MA 02139, USA carreras@csail.mit.edu mcollins@csail.mit.edu Abstract This paper describes an empirical study of high-performance dependency parsers based on a semi-supervised learning ap- proach. We describe an extension of semi- supervised structured conditional models (SS-SCMs) to the dependency parsing problem, whose framework is originally proposed in (Suzuki and Isozaki, 2008). Moreover, we introduce two extensions re- lated to dependency parsing: The first ex- tension is to combine SS-SCMs with an- other semi-supervised approach, described in (Koo et al., 2008). The second exten- sion is to apply the approach to second- order parsing models, such as those de- scribed in (Carreras, 2007), using a two- stage semi-supervised learning approach. We demonstrate the effectiveness of our proposed methods on dependency parsing experiments using two widely used test collections: the Penn Treebank for En- glish, and the Prague Dependency Tree- bank for Czech. Our best results on test data in the above datasets achieve 93.79% parent-prediction accuracy for En- glish, and 88.05% for Czech. 1 Introduction Recent work has successfully developed depen- dency parsing models for many languages us- ing supervised learning algorithms (Buchholz and Marsi, 2006; Nivre et al., 2007). Semi-supervised learning methods, which make use of unlabeled data in addition to labeled examples, have the po- tential to give improved performance over purely supervised methods for dependency parsing. is often straightforward to obtain large amountsIt of unlabeled data, making semi-supervised ap- proaches appealing; previous work on semi- supervised methods for dependency parsing in- cludes (Smith and Eisner, 2007; Koo et al., 2008; Wang et al., 2008). In particular, Koo et al. (2008) describe a semi-supervised approach that makes use of clus- ter features induced from unlabeled data, and gives state-of-the-art results on the widely used depen- dency parsing test collections: the Penn Tree- bank (PTB) for English and the Prague Depen- dency Treebank (PDT) for Czech. This is a very simple approach, but provided significant perfor- mance improvements comparing with the state- of-the-art supervised dependency parsers such as (McDonald and Pereira, 2006). This paper introduces an alternative method for semi-supervised learning for dependency parsing. Our approach basically follows a framework pro- posed in (Suzuki and Isozaki, 2008). We extend it for dependency parsing, which we will refer to as a Semi-supervised Structured Conditional Model (SS-SCM). In this framework, a structured condi- tional model is constructed by incorporating a se- ries of generative models, whose parameters are estimated from unlabeled data. This paper de- scribes a basic method for learning within this ap- proach, and in addition describes two extensions. The first extension is to combine our method with the cluster-based semi-supervised method of (Koo et al., 2008). The second extension is to apply the approach to second-order …

Content truncated to 500 words