Data Augmentation via Subtree Swapping for Dependency Parsing of Low-Resource Languages

Proceedings of the 28th International Conference on Computational Linguistics, pages 3818–3830 Barcelona, Spain (Online), December 8-13, 2020 3818 Data Augmentation via Subtree Swapping for Dependency Parsing of Low-Resource Languages Mathieu Dehouck Carlos Gómez-Rodríguez Universidade da Coruña, CITIC FASTPARSE Lab, LyS Research Group, Departamento de Ciencias de la Computación y Tecnologías de la Información Campus Elviña, s/n, 15071 A Coruña, Spain {mathieu.dehouck,carlos.gomez}@udc.es Abstract The lack of annotated data is a big issue for building reliable NLP systems for most of the world’s languages. But this problem can be alleviated by automatic data generation. In this paper, we present a new data augmentation method for artificially creating new dependency-annotated sen- tences. The main idea is to swap subtrees between annotated sentences while enforcing strong constraints on those trees to ensure maximal grammaticality of the new sentences. We also pro- pose a method to perform low-resource experiments using resource-rich languages by mimicking low-resource languages by sampling sentences under a low-resource distribution. In a series of experiments, we show that our newly proposed data augmentation method outperforms previous proposals using the same basic inputs. 1 Introduction Data sparsity has been a problem since the beginning of natural language processing. Neural networks have not solved it and have made it even more visible with their hunger for data. Hence a revived interest in data augmentation, since artificially created annotation can prove very useful to tackle the lack of manually annotated training data. Many approaches have been proposed to perform data augmentation. Some rely on external resources, such as unannotated raw text in order to iteratively increase their training data with automatically anno- tated examples (McClosky et al., 2006; Yu and Bohnet, 2015). However, this is an error-prone method and useful annotations must be separated from harmful ones. Other proposals instead rely solely on available annotated data in order to generate new examples (Şahin and Steedman, 2018). In this paper, we present a language-agnostic approach to the automatic generation of dependency- annotated sentences. Our method essentially swaps compatible subtrees from different sentences in order to generate new annotated sentences. By enforcing a number of constraints on the subtrees to be swapped, we avoid to generate too ungrammatical sentences. Furthermore, contrary to previous work that kept the syntactic structure of sentences or impoverished it, our method injects structures from other sentences, so it can introduce more syntactic complexity in the generated sentences. In order to assess the potential of our new data augmentation method for low-resource languages in a way that is independent from the specific sentences in low-resource treebanks, we propose a method to mimic low-resource language data using high-resource language data. By sampling sentences from high- resource languages using a low-resource language distribution (over sentence length), we can perform the same experiment several times in a more faithful low-resource setting, while dampening the role played by each individual sentence on the final results, which can be prominent in very low-resource settings. While there are many methods to parsing low-resource languages, such as unsupervised parsing or …

Content truncated to 500 words