Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers

Proceedings of the 28th International Conference on Computational Linguistics, pages 3886–3898 Barcelona, Spain (Online), December 8-13, 2020 3886 Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers Robert Litschko1 Ivan Vuli´c2 ˇZeljko Agi´c3 Goran Glavaˇs1 1 Data and Web Science Group, University of Mannheim, Germany 2 Language Technology Lab, University of Cambridge, UK 3 Unity Technologies, Copenhagen, Denmark {litschko,goran}@informatik.uni-mannheim.de iv250@cam.ac.uk zeljko.agic@unity.com Abstract Current methods of cross-lingual parser transfer focus on predicting the best parser for a low- resource target language globally, that is, “at treebank level”. In this work, we propose and argue for a novel cross-lingual transfer paradigm: instance-level parser selection (ILPS), and present a proof-of-concept study focused on instance-level selection in the framework of delexicalized parser transfer. Our work is motivated by an empirical observation that different source parsers are the best choice for different Universal POS-sequences (i.e., UPOS sentences) in the target language. We then propose to predict the best parser at the instance level. To this end, we train a supervised regression model, based on the Transformer architecture, to predict parser accuracies for individual POS-sequences. We compare ILPS against two strong single-best parser selection baselines (SBPS): (1) a model that compares POS n-gram distributions between the source and target languages (KL) and (2) a model that selects the source based on the similarity between manually created language vectors encoding syntactic properties of languages (L2V). The results from our extensive evaluation, coupling 42 source parsers and 20 diverse low-resource test languages, show that ILPS outperforms KL and L2V on 13/20 and 14/20 test languages, respectively. Further, we show that by predicting the best parser “at treebank level” (SBPS), using the aggregation of predictions from our instance-level model, we outperform the same baselines on 17/20 and 16/20 test languages. 1 Introduction Despite recent evidence that, for pretrained transformers like BERT (Devlin et al., 2019), explicit syntax contributes negligibly to better natural language understanding (Kuncoro et al., 2020; Glavaˇs and Vuli´c, 2020), proper language-specific automated syntactic analyses are still paramount in numerous tasks in computational linguistics, yet unavailable for 99% of the world’s languages due to the lack of respective treebanks. A major goal and promise of cross-lingual transfer in NLP is to transfer language technology to as many languages as possible (O’Horan et al., 2016; Ponti et al., 2019; Joshi et al., 2020; Lauscher et al., 2020). Therefore, cross-lingual transfer of dependency parsers has profiled as the most viable strategy to use parsing technology in resource-low languages (McDonald et al., 2011; Søgaard, 2011; Kondratyuk and Straka, 2019; ¨Ust¨un et al., 2020). Delexicalized transfer is conceptually the least demanding option in terms of language-specific resource requirements. The only provision, in order to transfer the parser trained on a delexicalized treebank of a resource-rich language, is a POS tagger in a low-resource target language based on the Universal POS (UPOS) tagset (Petrov et al., 2012). Delexicalized transfer is nowadays used primarily as a simple yet competitive baseline for more sophisticated transfer models when porting parsing technology in a new …

Content truncated to 500 words