Low-Resource Syntactic Transfer with Unsupervised Source Reordering

Proceedings of NAACL-HLT 2019, pages 3845–3856 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3845 Low-Resource Syntactic Transfer with Unsupervised Source Reordering Mohammad Sadegh Rasooli Facebook AI Menlo Park, CA, USA rasooli@fb.com Michael Collins Department of Computer Science Columbia University mcollins@cs.columbia.edu Abstract We describe a cross-lingual transfer method for dependency parsing that takes into account the problem of word order differences between source and target languages. Our model only relies on the Bible, a considerably smaller par- allel data than the commonly used parallel data in transfer methods. We use the concate- nation of projected trees from the Bible cor- pus, and the gold-standard treebanks in multi- ple source languages along with cross-lingual word representations. We demonstrate that re- ordering the source treebanks before training on them for a target language improves the ac- curacy of languages outside the European lan- guage family. Our experiments on 68 tree- banks (38 languages) in the Universal Depen- dencies corpus achieve a high accuracy for all languages. Among them, our experiments on 16 treebanks of 12 non-European languages achieve an average UAS absolute improve- ment of 3.3% over a state-of-the-art method. 1 Introduction There has recently been a great deal of interest in cross-lingual transfer of dependency parsers, for which a parser is trained for a target language of interest using treebanks in other languages. Cross- lingual transfer can eliminate the need for the ex- pensive and time-consuming task of treebank an- notation for low-resource languages. Approaches include annotation projection using parallel data sets (Hwa et al., 2005; Ganchev et al., 2009), direct model transfer through learning of a delexicalized model from other treebanks (Zeman and Resnik, 2008; T¨ackstr¨om et al., 2013), treebank transla- tion (Tiedemann et al., 2014), using synthetic tree- banks (Tiedemann and Agi´c, 2016; Wang and Eis- ner, 2016), using cross-lingual word representa- tions (T¨ackstr¨om et al., 2012; Guo et al., 2016; Rasooli and Collins, 2017) and using cross-lingual dictionaries (Durrett et al., 2012). Recent results from Rasooli and Collins (2017) have shown accuracies exceeding 80% on unla- beled attachment accuracy (UAS) for several Eu- ropean languages.1 However non-European lan- guages remain a significant challenge for cross- lingual transfer. One hypothesis, which we inves- tigate in this paper, is that word-order differences between languages are a significant challenge for cross-lingual transfer methods. The main goal of our work is therefore to reorder gold-standard source treebanks to make those treebanks syntac- tically more similar to the target language of in- terest. We use two different approaches for source treebank reordering: 1) reordering based on dom- inant dependency directions according to the pro- jected dependencies, 2) learning a classifier on the alignment data. We show that an ensemble of these methods with the baseline method leads to higher performance for the majority of datasets in our experiments. We show particularly significant improvements for non-European languages.2 The main contributions of this work are as fol- lows: • We propose two different syntactic reorder- ing methods based on the …

Content truncated to 500 words