PPT: Parsimonious Parser Transfer for Unsupervised Cross-Lingual Adaptation

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2907–2918 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2907 PPT: Parsimonious Parser Transfer for Unsupervised Cross-Lingual Adaptation Kemal Kurniawan1 Lea Frermann1 Philip Schulz2∗ Trevor Cohn1 1School of Computing and Information Systems, University of Melbourne 2Amazon Research kemal.kurniawan@student.unimelb.edu.au lea.frermann@unimelb.edu.au phschulz@amazon.com tcohn@unimelb.edu.au Abstract Cross-lingual transfer is a leading technique for parsing low-resource languages in the ab- sence of explicit supervision. Simple ‘direct transfer’ of a learned model based on a multi- lingual input encoding has provided a strong benchmark. This paper presents a method for unsupervised cross-lingual transfer that im- proves over direct transfer systems by using their output as implicit supervision as part of self-training on unlabelled text in the target language. The method assumes minimal re- sources and provides maximal flexibility by (a) accepting any pre-trained arc-factored de- pendency parser; (b) assuming no access to source language data; (c) supporting both pro- jective and non-projective parsing; and (d) sup- porting multi-source transfer. With English as the source language, we show significant im- provements over state-of-the-art transfer mod- els on both distant and nearby languages, de- spite our conceptually simpler approach. We provide analyses of the choice of source lan- guages for multi-source transfer, and the ad- vantage of non-projective parsing. Our code is available online.1 1 Introduction Recent progress in natural language processing (NLP) has been largely driven by increasing amounts and size of labelled datasets. The ma- jority of the world’s languages, however, are low- resource, with little to no labelled data avail- able (Joshi et al., 2020). Predicting linguistic labels, such as syntactic dependencies, underlies many downstream NLP applications, and the most ef- fective systems rely on labelled data. Their lack hinders the access to NLP technology in many languages. One solution is cross-lingual model ∗Work done outside Amazon. 1https://github.com/kmkurn/ ppt-eacl2021 xi den ich heute gesehen habe who I today seen have ˜A(xi) den habe 0.7 obj ich habe 0.1 nsubj ich gesehen 0.5 nsubj den gesehen 0.07 obj heute gesehen 0.2 advmod gesehen habe 0.02 aux ˜Y(xi) den ich heute gesehen habe who I today seen have nsubj advmod obj aux obj advmod nsubj aux {xi, ˜Y(xi)} sθ0(x, h, m) Pθ0(yi|xi) Figure 1: Illustration of our technique. For a target language sentence (xi), a source parser Pθ0 predicts a set of candidate arcs ˜A(xi) (subset shown in the fig- ure), and parses ˜Y (xi). The highest scoring parse is shown on the bottom (green), and the true gold parse (unknown to the parser) on top (red). A target language parser Pθ is then fine-tuned on a data set of ambigu- ously labelled sentences {xi, ˜Y (xi)}. transfer, which adapts models trained on high- resource languages to low-resource ones. This pa- per presents a flexible framework for cross-lingual transfer of syntactic dependency parsers which can leverage any pre-trained arc-factored dependency parser, and assumes no access to labelled target language data. One straightforward method of cross-lingual parsing is direct transfer. …

Content truncated to 500 words