Cross-Lingual Dependency Parsing with Late Decoding for Truly Low-Resource Languages

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 220–229, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Cross-Lingual Dependency Parsing with Late Decoding for Truly Low-Resource Languages Michael Sejr Schlichtkrull University of Amsterdam∗ m.s.schlichtkrull@uva.nl Anders Søgaard University of Copenhagen soegaard@di.ku.dk Abstract In cross-lingual dependency annotation projection, information is often lost dur- ing transfer because of early decoding. We present an end-to-end graph-based neural network dependency parser that can be trained to reproduce matrices of edge scores, which can be directly pro- jected across word alignments. We show that our approach to cross-lingual depen- dency parsing is not only simpler, but also achieves an absolute improvement of 2.25% averaged across 10 languages com- pared to the previous state of the art. 1 Introduction Dependency parsing is an integral part of many natural language processing systems. However, most research into dependency parsing has fo- cused on learning from treebanks, i.e. collec- tions of manually annotated, well-formed syntac- tic trees. In this paper, we develop and evaluate a graph-based parser which does not require the training data to be well-formed trees. We show that such a parser has an important application in cross-lingual learning. Annotation projection is a method for develop- ing parsers for low-resource languages, relying on aligned translations from resource-rich source lan- guages into the target language, rather than lin- guistic resources such as treebanks or dictionar- ies. The Bible has been translated completely into 542 languages, and partially translated into a fur- ther 2344 languages. As such, the assumption that we have access to parallel Bible data is much less constraining than the assumption of access to linguistic resources. Furthermore, for truly low- resource languages, relying upon the Bible scales ∗Work done while at the University of Copenhagen. better than relying on less biased data such as the EuroParl corpus. In Agi´c et al. (2016), a projection scheme is proposed wherein labels are collected from many sources, projected into a target language, and then averaged. Crucially, the paper demonstrates how projecting and averaging edge scores from a graph-based parser before decoding improves per- formance. Even so, decoding is still a requirement between projecting labels and retraining from the projected data, since their parser (TurboParser) re- quires well-formed input trees. This introduces a potential source of noise and loss of information that may be important for finding the best target sentence parse. Our approach circumvents the need for decod- ing prior to training, thereby surpassing a state- of-the-art dependency parser trained on decoded multi-source annotation projections as done by Agi´c et al. We first evaluate the model across sev- eral languages, demonstrating results comparable to the state of the art on the Universal Dependen- cies (McDonald et al., 2013) dataset. Then, we evaluate the same model by inducing labels from cross-lingual multi-source annotation projection, comparing the performance of a model with early decoding to a model with late decoding. Contributions We present a novel end-to-end neural graph-based …

Content truncated to 500 words