Cross-lingual Transfer for Unsupervised Dependency Parsing Without Parallel Data

Proceedings of the 19th Conference on Computational Language Learning, pages 113–122, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics Cross-lingual Transfer for Unsupervised Dependency Parsing Without Parallel Data Long Duong,12 Trevor Cohn,1 Steven Bird,1 and Paul Cook3 1Department of Computing and Information Systems, University of Melbourne 2National ICT Australia, Victoria Research Laboratory 3Faculty of Computer Science, University of New Brunswick lduong@student.unimelb.edu.au {t.cohn,sbird}@unimelb.edu.au paul.cook@unb.ca Abstract Cross-lingual transfer has been shown to produce good results for dependency pars- ing of resource-poor languages. Although this avoids the need for a target language treebank, most approaches have still used large parallel corpora. However, parallel data is scarce for low-resource languages, and we report a new method that does not need parallel data. Our method learns syntactic word embeddings that generalise over the syntactic contexts of a bilingual vocabulary, and incorporates these into a neural network parser. We show empir- ical improvements over a baseline delex- icalised parser on both the CoNLL and Universal Dependency Treebank datasets. We analyse the importance of the source languages, and show that combining mul- tiple source-languages leads to a substan- tial improvement. 1 Introduction Dependency parsing is a crucial component of many natural language processing (NLP) systems for tasks such as relation extraction (Bunescu and Mooney, 2005), statistical machine transla- tion (Xu et al., 2009), text classification ( ¨Ozg¨ur and G¨ung¨or, 2010), and question answering (Cui et al., 2005). Supervised approaches to depen- dency parsing have been very successful for many resource-rich languages, where relatively large treebanks are available (McDonald et al., 2005a). However, for many languages, annotated tree- banks are not available, and are very costly to cre- ate (B¨ohmov´a et al., 2001). This motivates the development of unsupervised approaches that can make use of unannotated, monolingual data. How- ever, purely unsupervised approaches have rela- tively low accuracy (Klein and Manning, 2004; Gelling et al., 2012). Most recent work on unsupervised dependency parsing for low-resource languages has used the idea of delexicalized parsing and cross-lingual transfer (Zeman et al., 2008; Søgaard, 2011; Mc- Donald et al., 2011; Ma and Xia, 2014). In this setting, a delexicalized parser is trained on a resource-rich source language, and is then applied directly to a resource-poor target language. The only requirement here is that the source and tar- get languages are POS tagged must use the same tagset. This assumption is pertinent for resource- poor languages since it is relatively quick to man- ually POS tag the data. Moreover, there are many reports of high accuracy POS tagging for resource- poor languages (Duong et al., 2014; Garrette et al., 2013; Duong et al., 2013b). The cross-lingual delexicalized approach has been shown to signif- icantly outperform unsupervised approaches (Mc- Donald et al., 2011; Ma and Xia, 2014). Parallel data can be used to boost the perfor- mance of a cross-lingual parser (McDonald et al., 2011; Ma and Xia, 2014). However, parallel data may be hard to acquire for truly resource-poor lan- guages.1 Accordingly, we propose a method to improve the performance …

Content truncated to 500 words