Generating Artificial Texts as Substitution or Complement of Training Data

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4260–4269 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4260 Generating Artificial Texts as Substitution or Complement of Training Data Vincent Claveau∗, Antoine Chaffin∗,†, Ewa Kijak∗ ∗ IRISA - CNRS - Univ. Rennes IRISA, Campus de Beaulieu, 35042 Rennes, France † IMATAG 13 Rue Dupont-des-Loges, 35000 Rennes, France vincent.claveau@irisa.fr, antoine.chaffin@imatag.com, ewa.kijak@irisa.fr Abstract The quality of artificially generated texts has considerably improved with the advent of transformers. The question of using these models to generate learning data for supervised learning tasks naturally arises, especially when the original language resource cannot be distributed, or when it is small. In this article, this question is explored under 3 aspects: (i) are artificial data an efficient complement? (ii) can they replace the original data when those are not available or cannot be distributed for confidentiality reasons? (iii) can they improve the explainability of classifiers? Different experiments are carried out on classification tasks — namely sentiment analysis on product reviews, Fake News detection and news categorization — using artificially generated data by fine-tuned GPT-2 models. The results show that such artificial data can be used in a certain extent but require pre-processing to significantly improve performance. We also show that bag-of-words approaches benefit the most from such data augmentation. Keywords: Text generation, Data augmentation, Classification, Categorization, Fake News, Sentiment Analysis 1. Introduction Even if text generation is not a new technology, re- cent neural approaches based on transformers offers good enough performance to be used in various con- texts (Vaswani et al., 2017). In this paper, we ex- plore the use of artificially generated texts for super- vised machine learning tasks within two different sce- narios: the artificial data is used as a complement of the original training dataset (for instance, to yield bet- ter performance) or the data is used as a substitute of the original data (for instance, when the original data cannot be shared because they contain confidential in- formation (Amin-Nejad et al., 2020)). The generation of these artificial texts is performed with a neural lan- guage model trained on the original training texts. In this paper, we show the interest of these scenarios with several text classification tasks, handling well written or noisy language: fake news detection, opinion min- ing and news categorization. Precisely, the main research questions studied in this paper are the following ones: 1. what is the interest of text generation to improve text classification (complement); 2. what is the interest of text generation to replace the original training data (substitution); 3. what is the interest of text generation for explain- able classifiers, based on bag-of-words represen- tation. In the remaining of the paper, after a presentation of related work in Section 2, we detail our classification approaches based on artificial text generation (Sec. 3). The tasks and experimental data are described in Sec- tion 4. The experiments and their results for each of our research questions are reported in …

Content truncated to 500 words