Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsing

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6547 - 6563 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsing Freda Shi Kevin Gimpel Karen Livescu Toyota Technological Institute at Chicago 6045 S Kenwood Ave, Chicago, IL, USA, 60637 {freda, kgimpel, klivescu}@ttic.edu Abstract We present substructure distribution projection (SUBDP), a technique that projects a distribu- tion over structures in one domain to another, by projecting substructure distributions sepa- rately. Models for the target domain can then be trained, using the projected distributions as soft silver labels. We evaluate SUBDP on zero- shot cross-lingual dependency parsing, taking dependency arcs as substructures: we project the predicted dependency arc distributions in the source language(s) to target language(s), and train a target language parser on the re- sulting distributions. Given an English tree- bank as the only source of human supervision, SUBDP achieves better unlabeled attachment score than all prior work on the Universal De- pendencies v2.2 (Nivre et al., 2020) test set across eight diverse target languages, as well as the best labeled attachment score on six lan- guages. In addition, SUBDP improves zero- shot cross-lingual dependency parsing with very few (e.g., 50) supervised bitext pairs, across a broader range of target languages. 1 Introduction Zero-shot cross-lingual dependency parsing is the task that requires prediction of dependency parses without seeing any parsing example in the target language; instead, the model may use annotated parses in other languages. A popular line of work is annotation projection: the parses generated by a source language dependency parser are projected into the target language, where the projected parses are then used to train a new parser. As illustrated in Figure 1b, most annotation projection methods typ- ically output partial hard dependency trees,1 where there either is or is not an arc between any pair of 1Throughout this paper, we refer to dependency parse trees with 0/1 arc and label probabilities, i.e., conventional depen- dency trees, as hard trees; in contrast, we refer to collections of per-word head distributions and per-arc label distributions with continuous probabilities as soft trees. I went to the book store 我 去 了 書店 I go (p. PT) book store nsubj (0.9) obl (1.0) compound (1.0) case (0.8) case (0.2) det (1.0) root (1) root (0.1) nsubj (0.9) obl (0.5) root (1) root (0.1) compound (0.5) p. PT: particle denoting past tense. (a) Soft tree projection with SUBDP. Best viewed in color. I went to the book store 我 去 了 書店 I go (p. PT) book store nsubj obl compound case det root nsubj root (b) Projection with only one-to-one alignments. Figure 1: Illustration of SUBDP (top) vs. a represen- tative of annotation projection (bottom; Lacroix et al., 2016). An English parse tree, labeled with the Uni- versal Dependencies conventions (Nivre et al., 2016, 2020), is projected to the parallel Chinese sentence. We denote dependency edges by arrows with the cor- responding arc probabilities (if applicable) …

Content truncated to 500 words