Effective Data Augmentation for Sentence Classification Using One VAE per Class

Proceedings of the 29th International Conference on Computational Linguistics, pages 3454–3464 October 12–17, 2022. 3454 Effective data augmentation for sentence classification using one VAE per class Piedboeuf, Frédéric and Langlais, Philippe DIRO, RALI Université de Montréal Montréal, Québec, Canada Abstract In recent years, data augmentation has become an important field of machine learning. While images can use simple techniques such as crop- ping or rotating, textual data augmentation needs more complex manipulations to ensure that the generated examples are useful. Varia- tional auto-encoders (VAE) and its conditional variant the Conditional-VAE (CVAE) are often used to generate new textual data, both relying on a good enough training of the generator so that it doesn’t create examples of the wrong class. In this paper, we explore a simpler way to use VAE for data augmentation: the train- ing of one VAE per class. We show on several dataset sizes, as well as on four different binary classification tasks, that it outperforms other generative data augmentation techniques. 1 Introduction Data augmentation (DA) has been shown to be an efficient technique for deep neural net- works (Ramirez Rochac et al., 2019), consisting in artificially creating new labelled examples and therefore inflating the size of the dataset. While a great number of data augmentation techniques have been developed in recent years, most of these re- quire external data, either in a database form (Wei and Zou, 2019), pretrained embeddings (Marivate and Sefara, 2020), or neural networks (Wu et al., 2019). Even if these techniques help improve clas- sifiers, they cannot be used in a variety of contexts, most notably in artificial intelligence for rare lan- guages, where there simply isn’t enough training data to pre-train efficient models (Feldman and Coto-Solano, 2020), or where the collection of la- belled data for fine-tuning is difficult. The ethi- cal importance of considering smaller communi- ties and other languages in machine learning re- search has been noted repeatedly (Bender et al., 2021; Fazelpour and De-Arteaga, 2022), and con- sequently, it is important to develop techniques that can work for languages other than English. In this paper, we take a special look at generative models for data augmentation, and more specifi- cally at Variational Auto-Encoders (VAE), for bi- nary classification tasks. Generative algorithms such as the VAE are especially interesting because they do not require external data and, therefore, can be used on a variety of domains. In particular, we show that using one separate VAE per class and generating new data by random sampling of the latent space is a very efficient way of inflating the size of a dataset. To the best of our knowledge, this technique has been considered only once before, in the context of balancing unbalanced datasets, and with somewhat disappointing results (Qiu et al., 2020). Contrary to them, we study the technique in the context of pure data augmentation, on more datasets and various starting sizes of datasets, and conclude that it is an efficient technique. We test two variations of this method, with …

Content truncated to 500 words