Text Style Transfer Back-Translation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7944–7959 July 9-14, 2023 ©2023 Association for Computational Linguistics Text Style Transfer Back-Translation Daimeng Wei∗, Zhanglin Wu∗, Hengchao Shang, Zongyao Li, Minghan Wang, Jiaxin Guo, Xiaoyu Chen, Zhengzhe Yu, Hao Yang Huawei Translation Service Center, Beijing, China {weidaimeng,wuzhanglin2,shanghengchao,lizongyao,wangminghan, guojiaxin1,chenxiaoyu35,yuzhengzhe,yanghao30}@huawei.com Abstract Back Translation (BT) is widely used in the field of machine translation, as it has been proved effective for enhancing transla- tion quality. However, BT mainly improves the translation of inputs that share a similar style (to be more specific, translation-like in- puts), since the source side of BT data is machine-translated. For natural inputs, BT brings only slight improvements and some- times even adverse effects. To address this issue, we propose Text Style Transfer Back Translation (TST BT), which uses a style trans- fer model to modify the source side of BT data. By making the style of source-side text more natural, we aim to improve the translation of natural inputs. Our experiments on various lan- guage pairs, including both high-resource and low-resource ones, demonstrate that TST BT significantly improves translation performance against popular BT benchmarks. In addition, TST BT is proved to be effective in domain adaptation so this strategy can be regarded as a general data augmentation method. Our train- ing code and text style transfer model are open- sourced.1 1 Introduction Works in neural machine translation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2016; Wu and et.al, 2016; Vaswani et al., 2017) greatly im- prove translation quality. However, current meth- ods generally require large amount of bilingual training data, which is a challenging and some- times impossible task. As obtaining monolingual data is much easier, researchers have long ex- ploited methods to enhance model performances using monolingual data, for example, language model fusion for phrase-based (Brants et al., 2007; Koehn, 2009) and neural machine translation (Gul- cehre et al., 2015, 2017), back translation (Sen- nrich et al., 2016), and dual learning (Cheng et al., *These authors contributed equally to this work. 1https://github.com/FrxxzHL/ssebt Figure 1: Bilingual and BT data used for English → German training. Nature indicates data generated by native speakers; HT indicates data generated by hu- man translators from another language, and MT indi- cates machine translation results. MT and HT styles are close, but far from Nature. 2016; He et al., 2016; Xia et al., 2017). The com- bination of such monolingual methods can further improve model performances. Back Translation (BT), a data augmentation method to generate synthetic parallel data by trans- lating content from target language back to source language, is widely used in the field of machine translation. BT has many variants (Sennrich et al., 2016; Edunov et al., 2018; Caswell et al., 2019) and each has own merits. In terms of text style, models that use BT are usually trained on three types of data. Real par- allel data constitutes the first two types: natural source with human-translated target (Nature → HT) or …

Content truncated to 500 words