Improving Low-resource RRG Parsing with Cross-lingual Self-training

Proceedings of the 29th International Conference on Computational Linguistics, pages 4360–4371 October 12–17, 2022. 4360 Improving Low-resource RRG Parsing with Cross-lingual Self-training Kilian Evang Laura Kallmeyer Jakub Waszczuk Kilu von Prince Tatiana Bladier Simon Petitjean Heinrich Heine University Düsseldorf, Germany first.last@hhu.de Abstract This paper considers the task of parsing low- resource languages in a scenario where parallel English data and also a limited seed of anno- tated sentences in the target language are avail- able, as for example in bootstrapping parallel treebanks. We focus on constituency parsing using Role and Reference Grammar (RRG), a theory that has so far been understudied in computational linguistics but that is widely used in typological research, i.e., in particu- lar in the context of low-resource languages. Starting from an existing RRG parser, we pro- pose two strategies for low-resource parsing: first, we extend the parsing model into a cross- lingual parser, exploiting the parallel data in the high-resource language and unsupervised word alignments by providing internal states of the source-language parser to the target-language parser. Second, we adopt self-training, thereby iteratively expanding the training data, starting from the seed, by including the most confident new parses in each round. Both in simulated scenarios and with a real low-resource language (Daakaka), we find substantial and complemen- tary improvements from both self-training and cross-lingual parsing. Moreover, we also ex- perimented with using gloss embeddings in ad- dition to token embeddings in the target lan- guage, and this also improves results. Finally, starting from what we have for Daakaka, we also consider parsing a related language (Dal- kalaen) where glosses and English translations are available but no annotated trees at all, i.e., a no-resource scenario wrt. syntactic annotations. We start with cross-lingual parser trained on Daakaka with glosses and use self-training to adapt it to Dalkalaen. The results are surpris- ingly good.1 1 Introduction Treebanks play an increasingly important role in typological research, where linguists are oftentimes 1Our experimental code is available at https:// gitlab.com/treegrasp/rrgproj2. SENTENCE CLAUSE CLAUSE CORE NP PRO ra 1p.in v.pre OPtns -t -DIST -v.pre NUC V warsyosi revere v , , CLAUSE CORE NUC V warsyosi revere v NP PRO nge 3s pron “We revered, revered him.” Figure 1: RRG annotation of a Daakaka sentence with glosses, parts of speech, and translation. Glosses: 1p.in– first person plural inclusive; DIST–distal TAM marker (for past and counterfactual contexts); 3s: third person singular faced with (primarily) oral low-resource languages and where grammatical frameworks such as Role and Reference Grammar (RRG; Van Valin and Foley, 1980; Van Valin, 2005) are a common choice (Toratani and González Vergara, 2020). RRG is a non-transformational linguistic theory strongly inspired by typological concerns. Its development was guided by the question of what a linguistic theory would “look like if it were based on the analysis of languages with diverse structures such as Lakhota, Tagalog and Dyirbal [...]?” (Van Valin, 2005, p. 1). RRG assumes constituency structures to be organized in layers, viz. nucleus (containing the predicate), core (containing the nucleus and …

Content truncated to 500 words