Cross-lingual Dependency Parsing Based on Distributed Representations

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1234–1244, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Cross-lingual Dependency Parsing Based on Distributed Representations Jiang Guo1∗, Wanxiang Che1, David Yarowsky2, Haifeng Wang3, Ting Liu1 1Center for Social Computing and Information Retrieval, Harbin Institute of Technology 2Center for Language and Speech Processing, Johns Hopkins University 3Baidu Inc., Beijing, China {jguo, car, tliu}@ir.hit.edu.cn yarowsky@jhu.edu, wanghaifeng@baidu.com Abstract This paper investigates the problem of cross-lingual dependency parsing, aim- ing at inducing dependency parsers for low-resource languages while using only training data from a resource-rich lan- guage (e.g. English). Existing approaches typically don’t include lexical features, which are not transferable across lan- guages. In this paper, we bridge the lex- ical feature gap by using distributed fea- ture representations and their composition. We provide two algorithms for inducing cross-lingual distributed representations of words, which map vocabularies from two different languages into a common vector space. Consequently, both lexical features and non-lexical features can be used in our model for cross-lingual transfer. Furthermore, our framework is able to in- corporate additional useful features such as cross-lingual word clusters. Our com- bined contributions achieve an average rel- ative error reduction of 10.9% in labeled attachment score as compared with the delexicalized parser, trained on English universal treebank and transferred to three other languages. It also significantly out- performs McDonald et al. (2013) aug- mented with projected cluster features on identical data. 1 Introduction Dependency Parsing has been one of NLP’s long- standing central problems. The majority of work on dependency parsing has been dedicated to resource-rich languages, such as English and Chi- nese. For these languages, there exist large-scale ∗This work was done while the author was visiting JHU. annotated treebanks that can be used for super- vised training of dependency parsers. However, for most of the languages in the world, there are few or even no labeled training data for parsing, and it is labor intensive and time-consuming to manually build treebanks for all languages. This fact has given rise to a number of research on un- supervised methods (Klein and Manning, 2004), annotation projection methods (Hwa et al., 2005), and model transfer methods (McDonald et al., 2011) for predicting linguistic structures. In this study, we focus on the model transfer methods, which attempt to build parsers for low-resource languages by exploiting treebanks from resource- rich languages. The major obstacle in transferring a parsing system from one language to another is the lex- ical features, e.g. words, which are not directly transferable across languages. To solve this prob- lem, McDonald et al. (2011) build a delexical- ized parser - a parser that only has non-lexical features. A delexicalized parser makes sense in that POS tag features are significantly predic- tive for unlabeled dependency parsing. How- ever, for labeled dependency parsing, especially for semantic-oriented dependencies like Stanford- type dependencies (De Marneffe et al., 2006; De Marneffe and Manning, 2008), these …

Content truncated to 500 words