Unsupervised Dependency Parsing with Transferring Distribution via Parallel Guidance and Entropy Regularization

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1337–1348, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Unsupervised Dependency Parsing with Transferring Distribution via Parallel Guidance and Entropy Regularization Xuezhe Ma Department of Linguistics University of Washington Seattle, WA 98195, USA xzma@uw.edu Fei Xia Department of Linguistics University of Washington Seattle, WA 98195, USA fxia@uw.edu Abstract We present a novel approach for induc- ing unsupervised dependency parsers for languages that have no labeled training data, but have translated text in a resource- rich language. We train probabilistic pars- ing models for resource-poor languages by transferring cross-lingual knowledge from resource-rich language with entropy reg- ularization. Our method can be used as a purely monolingual dependency parser, requiring no human translations for the test data, thus making it applicable to a wide range of resource-poor languages. We perform experiments on three Data sets — Version 1.0 and version 2.0 of Google Universal Dependency Treebanks and Treebanks from CoNLL shared-tasks, across ten languages. We obtain state- of-the art performance of all the three data sets when compared with previously studied unsupervised and projected pars- ing systems. 1 Introduction In recent years, dependency parsing has gained universal interest due to its usefulness in a wide range of applications such as synonym gener- ation (Shinyama et al., 2002), relation extrac- tion (Nguyen et al., 2009) and machine trans- lation (Katz-Brown et al., 2011; Xie et al., 2011). Several supervised dependency parsing algorithms (Nivre and Scholz, 2004; McDonald et al., 2005a; McDonald et al., 2005b; McDon- ald and Pereira, 2006; Carreras, 2007; Koo and Collins, 2010; Ma and Zhao, 2012; Zhang et al., 2013) have been proposed and achieved high pars- ing accuracies on several treebanks, due in large part to the availability of dependency treebanks in a number of languages (McDonald et al., 2013). However, the manually annotated treebanks that these parsers rely on are highly expensive to cre- ate, in particular when we want to build treebanks for resource-poor languages. This led to a vast amount of research on unsupervised grammar in- duction (Carroll and Charniak, 1992; Klein and Manning, 2004; Smith and Eisner, 2005; Cohen and Smith, 2009; Spitkovsky et al., 2010; Blun- som and Cohn, 2010; Mareˇcek and Straka, 2013; Spitkovsky et al., 2013), which appears to be a natural solution to this problem, as unsupervised methods require only unannotated text for training parsers. Unfortunately, the unsupervised gram- mar induction systems’ parsing accuracies often significantly fall behind those of supervised sys- tems (McDonald et al., 2011). Furthermore, from a practical standpoint, it is rarely the case that we are completely devoid of resources for most lan- guages. In this paper, we consider a practically moti- vated scenario, in which we want to build statisti- cal parsers for resource-poor target languages, us- ing existing resources from a resource-rich source language (like English).1 We assume that there are absolutely no labeled training data for the target language, but we have access to parallel data with a …

Content truncated to 500 words