AMR-DA: Data Augmentation by Abstract Meaning Representation

Findings of the Association for Computational Linguistics: ACL 2022, pages 3082 - 3098 May 22-27, 2022 c⃝2022 Association for Computational Linguistics AMR-DA: Data Augmentation by Abstract Meaning Representation Ziyi SHOU1 Yuxin JIANG2 Fangzhen LIN 1 1Department of Computer Science and Engineering 2DSA Thrust, Information Hub The Hong Kong University of Science and Technology {zshou,flin}@cse.ust.hk, yjiangcm@connect.ust.hk Abstract Abstract Meaning Representation (AMR) is a semantic representation for NLP/NLU. In this paper, we propose to use it for data augmen- tation in NLP. Our proposed data augmenta- tion technique, called AMR-DA, converts a sample sentence to an AMR graph, modifies the graph according to various data augmen- tation policies, and then generates augmenta- tions from graphs. Our method combines both sentence-level techniques like back translation and token-level techniques like EDA (Easy Data Augmentation). To evaluate the effec- tiveness of our method, we apply it to the En- glish tasks of semantic textual similarity (STS) and text classification. For STS, our experi- ments show that AMR-DA boosts the perfor- mance of the state-of-the-art models on sev- eral STS benchmarks. For text classification, AMR-DA outperforms EDA and AEDA and leads to more robust improvements.1 1 Introduction Data augmentation (DA) techniques automatically generate additional data from existing data set for training machine learning models. They are widely used in computer vision (see, e.g. Perez and Wang, 2017) and can boost the performance of the trained models. In NLP, DA methods can be roughly classified into token-level ones and sentence-level ones (Chen et al., 2021). Token-level DA methods generate new sample sentences from the original ones by changing some of their tokens (words). They in- clude the method in Zhang et al. (2015) that re- places some random tokens by their synonyms us- ing a thesaurus, the now widely used Easy Data Augmentation (EDA) methods in Wei and Zou (2019) that allow some random token insertion, deletion and swaps, and the more recent one in 1Codes will be at https://github.com/zzshou/ amr-data-augmentation Liu et al. (2020) that performs token replacement using their embeddings. One advantage of these token-level DA methods is that they are easy to implement. However, they can sometimes generate ill-formed or incoherent sentences as they do not take the sentence structures into account. In con- trast, sentence-level methods generate new sample sentences by modifying the whole original sen- tences. They typically work by having an encoder that converts the input sentence to an intermediate representation and a decoder that generates new sentences from the intermediate representations. For example, in back translation (Sennrich et al., 2016), the intermediate representation is a sentence in another natural language. In generation methods (Kumar et al., 2020; Yang et al., 2020), the interme- diate representation is a hidden state. One advan- tage of sentence-level DA methods is that they can preserve the semantics of the sentences. A major limitation of current sentence-level DA methods is that there is not much variation in the generated sentences as the intermediate representations used are not easily controllable (Li et al., 2021). For ex- …

Content truncated to 500 words