Distributed Word Representation Learning for Cross-Lingual Dependency Parsing

Proceedings of the Eighteenth Conference on Computational Language Learning, pages 119–129, Baltimore, Maryland USA, June 26-27 2014. c⃝2014 Association for Computational Linguistics Distributed Word Representation Learning for Cross-Lingual Dependency Parsing Min Xiao and Yuhong Guo Department of Computer and Information Sciences Temple University Philadelphia, PA 19122, USA {minxiao,yuhong}@temple.edu Abstract This paper proposes to learn language- independent word representations to ad- dress cross-lingual dependency parsing, which aims to predict the dependency parsing trees for sentences in the target language by training a dependency parser with labeled sentences from a source lan- guage. We first combine all sentences from both languages to induce real-valued distributed representation of words under a deep neural network architecture, which is expected to capture semantic similari- ties of words not only within the same lan- guage but also across different languages. We then use the induced interlingual word representation as augmenting features to train a delexicalized dependency parser on labeled sentences in the source language and apply it to the target sentences. To in- vestigate the effectiveness of the proposed technique, extensive experiments are con- ducted on cross-lingual dependency pars- ing tasks with nine different languages. The experimental results demonstrate the superior cross-lingual generalizability of the word representation induced by the proposed approach, comparing to alterna- tive comparison methods. 1 Introduction With the rapid development of linguistic resources and tools in multiple languages, it is very im- portant to develop cross-lingual natural language processing (NLP) systems. Cross-lingual depen- dency parsing is the task of inferring dependency trees for observed sentences in a target language where there are few or no labeled training sen- tences by using a dependency parser trained on a large amount of sentences with annotated de- pendency trees in a source language (Durrett et al., 2012; McDonald et al., 2011; Zhao et al., 2009). Cross-lingual dependency parsing is pop- ularly studied in natural language processing area as it can greatly reduce the expensive manual an- notation effort in the target language by exploit- ing the dependency annotations from a source lan- guage (Durrett et al., 2012; McDonald et al., 2011; T¨ackstr¨om et al., 2012). One fundamental challenge of cross-lingual de- pendency parsing stems from the word-level rep- resentation divergence across languages. Since sentences in different languages are expressed using different vocabularies, if we train a de- pendency parser on the word-level features of sentences from a source language, it will fail to parse the sentences in a different target lan- guage. A variety of work in the literature has at- tempted to bridge the word-level representation di- vergence across languages. One intuitive method delexicalizes the dependency parser by replac- ing the language-specific word-level features with language-independent features such as universal part-of-speech tags (Petrov et al., 2012). With the universal POS tag features, this method provides a possible way to transfer dependency parsing in- formation from the source language to the target language and has demonstrated some good empir- ical results (McDonald et al., 2011). However, the number of universal POS tags is small, which …

Content truncated to 500 words