TreeSwap: Data Augmentation for Machine Translation via Dependency Subtree Swapping

Proceedings of Recent Advances in Natural Language Processing, pages 759–768 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_082 759 TreeSwap: Data Augmentation for Machine Translation via Dependency Subtree Swapping Attila Nagy1, Dorina Lakatos1,2, Botond Barta1,2, and Judit ´Acs2 1Department of Automation and Applied Informatics Budapest University of Technology and Economics attila.nagy234@gmail.com 2Institute for Computer Science and Control {botondbarta, dorinalakatos, acsjudit}@sztaki.hu Abstract Data augmentation methods for neural machine translation are particularly useful when limited amount of training data is available, which is often the case when dealing with low-resource languages. We introduce a novel augmenta- tion method, which generates new sentences by swapping objects and subjects across bisen- tences. This is performed simultaneously based on the dependency parse trees of the source and target sentences. We name this method TreeSwap. Our results show that TreeSwap achieves consistent improvements over base- line models in 4 language pairs in both direc- tions on resource-constrained datasets. We also explore domain-specific corpora, but find that our method does not make significant improve- ments on law, medical and IT data. We report the scores of similar augmentation methods and find that TreeSwap performs comparably. We also analyze the generated sentences qualita- tively and find that the augmentation produces a correct translation in most cases. Our code is available on Github1. 1 Introduction Most Natural Language Processing (NLP) prob- lems are formulated as supervised learning tasks, where large amounts of data is required to train models. Collecting annotated datasets is often time- consuming and laborious, so this motivated a lot of work in NLP to create methods for generating synthetic data that improves the dataset used for training in both size and variety, ultimately lead- ing to more performant models (Feng et al., 2021). These Data Augmentation (DA) methods not only help in resource-constrained scenarios, but can also improve class imbalance (Chawla et al., 2002), mit- igate bias (Zhao et al., 2018), make the model more robust to out of distribution inputs (Yao et al., 2022) 1https://github.com/attilanagy234/ TreeSwap, last accessed on 31/07/23 or simply improve model accuracy. An efficient data augmentation method for any NLP task has two main objectives, which need to be balanced: the augmented data should be diverse enough, that it provides new information during training, but it should also be label-preserving to avoid inject- ing unwanted noise into the model. In machine translation, this means that our aim is to generate diverse sentence pairs from existing data such that the parallelism holds. In this paper, we propose TreeSwap, a data aug- mentation method for Neural Machine Translation (NMT) using dependency parsing. The core idea of TreeSwap is to find corresponding subtrees in the dependency parse trees of a translation pair and swap these to generate new data. As our augmenta- tion procedure is based on dependency parsing with some additional rules to improve grammatical and morphological correctness, the generated sentence pairs are semantically nonsensical in many cases. Using such nonsensical or nonce, but syntactically correct sentences as training data has been studied before and shown to …

Content truncated to 500 words