Visual Information Guided Zero-Shot Paraphrase Generation

Proceedings of the 29th International Conference on Computational Linguistics, pages 6530–6539 October 12–17, 2022. 6530 Visual Information Guided Zero-Shot Paraphrase Generation Zhe Lin and Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {linzhe,wanxiaojun}@pku.edu.cn Abstract Zero-shot paraphrase generation has drawn much attention as the large-scale high-quality paraphrase corpus is limited. Back-translation, also known as the pivot-based method, is typi- cal to this end. Several works leverage different information as “pivot” such as language, se- mantic representation and so on. In this paper, we explore using visual information such as im- age as the “pivot” of back-translation. Different with the pipeline back-translation method, we propose visual information guided zero-shot paraphrase generation (ViPG) based only on paired image-caption data. It jointly trains an image captioning model and a paraphrasing model and leverage the image captioning model to guide the training of the paraphrasing model. Both automatic evaluation and human evalua- tion show our model can generate paraphrase with good relevancy, fluency and diversity, and image is a promising kind of pivot for zero-shot paraphrase generation. 1 Introduction Paraphrase generation is a long-standing problem for natural language processing that aims to rewrite a text in other forms while preserving its original semantics. Paraphrase generation has many appli- cations in other down-stream tasks, such as ma- chine translation (Mehdizadeh Seraj et al., 2015), semantic parsing (Berant and Liang, 2014) and so on. With the development of supervised seq2seq generation, most paraphrase systems depend on the large-scale aligned paraphrase corpora to train the seq2seq model. This leads to the fact that the quality of aligned corpora is extremely impor- tant for training a paraphrase system. However, high-quality paraphrase datasets are still lacking in many domains. To solve this problem, there are a few works focusing on zero-shot paraphrase gen- eration such as back-translation. Back-translation Caption of Figure (a): a person on skis makes her way through the snow. Caption of Figure (b): a person standing on skiis on the snowy slope. (b) (a) Figure 1: An example that similar images may have different captions. makes use of language as pivot and treats the back- translated text as the paraphrase of the original text. For example, Mallinson et al. (2017) leveraged multilingual neural machine translation to gener- ate paraphrase and Cai et al. (2021) proposed to employ semantic representation as the “pivot lan- guage” of back-translation to generate paraphrase. All these works show that back-translation can gen- erate high-quality paraphrase. Inspired by back-translation based paraphrase generation, we explore to leverage visual informa- 6531 tion (i.e. image in this study) to guide the zero-shot paraphrase generation as similar images or similar partial images may have different captions or de- scriptions that can be treated as paraphrases. Figure 1 shows an example. A naive method is that we can use image as the “pivot language” and generate paraphrase by back-translation with a text-to-image model and an image-to-text model. Unfortunately, text-to-image generation is still …

Content truncated to 500 words