Learning to Selectively Learn for Weakly Supervised Paraphrase Generation with Model-based Reinforcement Learning
21356Authors
nan
Abstract
AbstractParaphrase generation is an important language generation task attempting to interpret user intents and systematically generate new phrases of identical meanings to the given ones. However, the effectiveness of paraphrase generation is constrained by the access to the golden labeled data pairs where both the amount and the quality of the training data pairs are restricted. In this paper, we propose a new weakly supervised paraphrase generation approach that extends the success of a recent work that leverages reinforcement learning for effective model training with data selection. While data selection is privileged for the target task which has noisy data, developing a reinforced selective learning regime faces several unresolved challenges. In this paper, we carry on important discussions about the above problem and present a new model that could partially overcome the discussed issues with a model-based planning feature and a reward normalization feature. We perform extensive evaluation on four weakly supervised paraphrase generation tasks where the results show that our method could significantly improve the state-of-the-art performance on the evaluation datasets.
Keywords
Full Content
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 116 - 126 July 10-15, 2022 ©2022 Association for Computational Linguistics Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks Anton Chernyavskiy1, Dmitry Ilvovsky1, Pavel Kalinin2, Preslav Nakov3 1HSE University, Russia 2Yandex, Russia 3Qatar Computing Research Institute, HBKU, Qatar {acherniavskii, dilvovsky}@hse.ru kalinin.pavel.v@gmail.com pnakov@hbku.edu.qa Abstract The use of contrastive loss for representation learning has become prominent in computer vision, and it is now getting attention in Natu- ral Language Processing (NLP). Here, we ex- plore the idea of using a batch-softmax con- trastive loss when fine-tuning large-scale pre- trained transformer models to learn better task- specific sentence embeddings for pairwise sen- tence scoring tasks. We introduce and study a number of variations in the calculation of the loss as well as in the overall training proce- dure; in particular, we find that a special data shuffling can be quite important. Our experi- mental results show sizable improvements on a number of datasets and pairwise sentence scor- ing tasks including classification, ranking, and regression. Finally, we offer detailed analy- sis and discussion, which should be useful for researchers aiming to explore the utility of con- trastive loss in NLP. 1 Introduction Recent years have seen a revolution in Natural Lan- guage Processing (NLP) thanks to the advances in machine learning. A lot of attention has been paid to architectures, especially for deep learning, as well as to loss functions. Notably, loss functions based on similar ideas were proposed in unrelated papers in different machine learning fields under different names. This can cause difficulties when solving new problems or when designing new ex- periments based on previous results. To a greater extent, this applies to “universal” loss functions, which can be applied in different machine learning areas and tasks such as Computer Vision (CV), Rec- ommendation Systems, and NLP. An example of such loss function is the batch-softmax contrastive (BSC) loss, which we will discuss below. For many NLP tasks, it is important to obtain representations of sentences for semantic matching problems, since they can be used for further analysis, e.g., for find- ing the best answer to a question. Sentence BERT is a recent popular approach for this (Reimers and Gurevych, 2019): it can be trained with different loss functions, and we show that the choice of a loss function is impor- tant. Moreover, we show that it will not be optimal to take the “standard” batch-softmax contrastive loss, which is used for training SimCSE (Gao et al., 2021), a recent alternative to Sentence BERT, and we suggest ways to improve its efficiency. Our contributions can be summarized as follows: • We study the use of a batch-softmax con- trastive loss for fine-tuning large-scale trans- formers to learn better task-specific embed- dings for pairwise sentence scoring tasks. • We introduce and study a number of novel variations in the calculation of the loss such as symmetrization, incorporating labeled neg- atives, aligning scores on …
Content truncated to 500 words
Labeled Text
AbstractParaphrase generation is an important language generation task attempting to interpret user intents and systematically generate new phrases of identical meanings to the given ones. However, the effectiveness of paraphrase generation is constrained by the access to the golden labeled data pairs where both the amount and the quality of the training data pairs are restricted. In this paper, we propose a new weakly supervised paraphrase generation approach that extends the success of a recent work that leverages reinforcement learning for effective model training with data selection. While data selection is privileged for the target task which has noisy data, developing a reinforced selective learning regime faces several unresolved challenges. In this paper, we carry on important discussions about the above problem and present a new model that could partially overcome the discussed issues with a model-based planning feature and a reward normalization feature. We perform extensive evaluation on four weakly supervised paraphrase generation tasks where the results show that our method could significantly improve the state-of-the-art performance on the evaluation datasets.
Publication Details
- Paper ID:
- 21356
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan