Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7439 - 7468 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings Kalpesh Krishna♠ ∗ Deepak Nathani♦ Xavier Garcia♦ Bidisha Samanta♦ Partha Talukdar♦ ♠University of Massachusetts Amherst, ♦Google Research kalpesh@cs.umass.edu {xgarcia, dnathani, bidishasamanta, partha}@google.com Abstract Style transfer is the task of rewriting a sen- tence into a target style while approximately preserving content. While most prior litera- ture assumes access to a large style-labelled corpus, recent work (Riley et al., 2021) has at- tempted “few-shot” style transfer using just 3- 10 sentences at inference for style extraction. In this work, we study a relevant low-resource setting: style transfer for languages where no style-labelled corpora are available. We notice that existing few-shot methods perform this task poorly, often copying inputs verbatim. We push the state-of-the-art for few-shot style transfer with a new method modeling the stylistic difference between paraphrases. When compared to prior work, our model achieves 2-3x better performance in formal- ity transfer and code-mixing addition across seven languages. Moreover, our method is bet- ter at controlling the style transfer magnitude using an input scalar knob. We report promis- ing qualitative results for several attribute transfer tasks (sentiment transfer, simplifica- tion, gender neutralization, text anonymiza- tion) all without retraining the model. Finally, we find model evaluation to be difficult due to the lack of datasets and metrics for many languages. To facilitate future research we crowdsource formality annotations for 4000 sentence pairs in four Indic languages, and use this data to design our automatic evaluations.1 1 Introduction Style transfer is a natural language generation task in which input sentences need to be re-written into a target style, while preserving semantics. It has many applications such as writing assistance (Hei- dorn, 2000), controlling generation for attributes 1Please visit the project page for the paper resources: https://martiansideofthemoon.github.io/ 2022/03/03/acl22.html. *Work done during a Google Research India internship. DiffUR Style Vector Extractor - Target (Formal) It is certainly amongst my favorites. Source (Informal) Its def one of my favs Style Vector Extractor अपनी वाली जॉब मु झे मत बताओ. (don’t tell me about your job) transfer amount 𝜆 आपकी2 �नयु ��त3 के बारे में मु झे ना बताएं ।4 अपनी वाली नौकरी1 मु झे मत बताओ। 𝜆 = 0.5 𝜆 = 1.5 honorifics,2,4 Sanskrit3 / Persian1 words for “job” in formal Hindi Figure 1: An illustration of our few-shot style trans- fer system during inference. Our model extracts style vectors from exemplar English sentences as input (in this case formal/informal sentences) and uses their vec- tor difference to guide style transfer in other languages (Hindi). λ is used to control the magnitude of transfer: in this example our model produces more high Sanskrit words & honorifics (more formal) with higher λ. like simplicity, formality or persuasion (Xu et al., 2015; Smith et al., 2020; Niu and Carpuat, 2020), data augmentation (Xie et al., 2020; Lee et al., 2021), and author …

Content truncated to 500 words