Parallel Data Augmentation for Formality Style Transfer

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3221–3228 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3221 Parallel Data Augmentation for Formality Style Transfer Yi Zhang1∗, Tao Ge2, Xu Sun1 1MOE Key Lab of Computational Linguistics, School of EECS, Peking University 2Microsoft Research Asia, Beijing, China {zhangyi16,xusun}@pku.edu.cn tage@microsoft.com Abstract The main barrier to progress in the task of Formality Style Transfer is the inadequacy of training data. In this paper, we study how to augment parallel data and propose novel and simple data augmentation methods for this task to obtain useful sentence pairs with easily accessible models and systems. Experiments demonstrate that our augmented parallel data largely helps improve formality style transfer when it is used to pre-train the model, lead- ing to the state-of-the-art results in the GYAFC benchmark dataset1. 1 Introduction Formality style transfer (FST) is defined as the task of automatically transforming a piece of text in one particular formality style into another (Rao and Tetreault, 2018). For example, given an in- formal sentence, FST aims to preserve the style- independent content and output a formal sentence. Previous work tends to leverage neural networks (Xu et al., 2019; Niu et al., 2018; Wang et al., 2019) such as seq2seq models to address this challenge due to their powerful capability and large improve- ment over the traditional rule-based approaches (Rao and Tetreault, 2018). However, the perfor- mance of the neural network approaches is still limited by the inadequacy of training data: the pub- lic parallel corpus for FST training – GYAFC (Rao and Tetreault, 2018) – contains only approximately 100K sentence pairs, which can hardly satiate the neural models with millions of parameters. To tackle the data sparsity problem for FST, we propose to augment parallel data with three specific data augmentation methods to help improve the model’s generalization ability and reduce the over- fitting risk. Besides applying the widely used back ∗Work done during the internship at Microsoft Research. 1Our augmented data is available at https://github. com/lancopku/Augmented_Data_for_FST FST (test instance) Input (informal) I dunno, even if she like you, and then she 'll prob. Reference (formal) I don't know. She probably will if she likes you. F-Dis Source I dunno... good luck. French Je ne sais pas... bonne chance. Target I don't know ... Good luck. M-Task Source I think she like cat too. Target I think she likes cat too. MT MT Figure 1: An example that Formality Style Transfer (FST) benefits from data augmented via formality discrimination (F-Dis) and multi-task transfer (M- Task). The mapping knowledge indicated by the color (blue→pink) in FST test instance occur in the pairs aug- mented by F-Dis and M-Task. F-Dis identifies useful sentence pairs from paraphrased sentence pairs gener- ated by cross-lingual MT, while M-Task utilizes train- ing data from GEC to help formality improvement. translation (BT) method (Sennrich et al., 2016a) in Machine Translation (MT) to FST, our data aug- mentation methods include formality discrimina- tion (F-Dis) and multi-task transfer …

Content truncated to 500 words