Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2164–2171 May 2-6, 2023 ©2023 Association for Computational Linguistics Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing Jivnesh Sandhan1, Laxmidhar Behera1,3 and Pawan Goyal2 1IIT Kanpur, 2IIT Kharagpur, 3IIT Mandi jivnesh@iitk.ac.in,pawang@cse.iitkgp.ac.in Abstract In this work, we focus on low-resource depen- dency parsing for multiple languages. Several strategies are tailored to enhance performance in low-resource scenarios. While these are well-known to the community, it is not triv- ial to select the best-performing combination of these strategies for a low-resource language that we are interested in, and not much atten- tion has been given to measuring the efficacy of these strategies. We experiment with 5 low- resource strategies for our ensembled approach on 7 Universal Dependency (UD) low-resource languages. Our exhaustive experimentation on these languages supports the effective improve- ments for languages not covered in pretrained models. We show a successful application of the ensembled system on a truly low-resource language Sanskrit.1 1 Introduction Recently, the supervised learning paradigm has dramatically increased the state-of-the-art perfor- mance for the dependency parsing task for resource- rich languages (Chen and Manning, 2014; Dyer et al., 2015; Kiperwasser and Goldberg, 2016; Dozat and Manning, 2017; Kulmizev et al., 2019). However, only a handful of resource-rich languages are able to take advantage, and many low-resource languages are far from these benefits (Joshi et al., 2020; More et al., 2019a; Zeman et al., 2018). In literature, several strategies have been pro- posed to enhance performance in low-resource sce- narios, such as data augmentation (¸Sahin and Steed- man, 2018; Gulordava et al., 2018), cross/mono- lingual pretraining (Conneau et al., 2020; Peters et al., 2018; Kondratyuk and Straka, 2019), sequen- tial transfer learning (Ruder et al., 2019), multi- task learning (Nguyen and Verspoor, 2018), cross- lingual transfer (Das and Sarkar, 2020) and self- training (Rotman and Reichart, 2019; Clark et al., 1The code and data are available at: https://github. com/Jivnesh/SanDP. 2018). However, not much attention has been given to measuring the efficacy of the existing low- resource strategies well-known to the community for low-resource dependency parsing (Vania et al., 2019). This is essential to assess their utility for low-resource languages (Hedderich et al., 2021) before inventing novel ways to tackle data sparsity. In this work, we systematically explore 5 prag- matic strategies for low-resource settings on 7 lan- guages. We experiment with low-resource strate- gies such as data augmentation, sequential trans- fer learning, cross/mono-lingual pretraining, multi- task learning and self-training. We investigate: (1) How is the trend in performance of each strategy across various languages? Whether the choice of best performing variant of each strategy is language dependent? (2) We integrate the best performing variant of each strategy and call the resulting sys- tem as the ensembled system. Do all the strate- gies contribute towards performance gain in the ensembled system? How well does this ensemble approach generalize across multiple low-resource languages? (3) How far can we push a purely …

Content truncated to 500 words