Do you have the right scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3436–3441 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3436 Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods Ning Miao Yuxuan Song Hao Zhou Lei Li ByteDance AI lab {miaoning,songyuxuan,zhouhao.nlp,lileilab}@bytedance.com Abstract It has been a common approach to pre-train a language model on a large corpus and fine- tune it on task-specific data. In practice, we observe that fine-tuning a pre-trained model on a small dataset may lead to over- and/or under-estimation problem. In this paper, we propose MC-Tailor, a novel method to alle- viate the above issue in text generation tasks by truncating and transferring the probability mass from over-estimated regions to under- estimated ones. Experiments on a variety of text generation datasets show that MC-Tailor consistently and significantly outperforms the fine-tuning approach. Our code is avail- able at https://github.com/NingMiao/ MC-tailor. 1 Introduction Recently, pre-trained language models (PLM), e.g. GPT-2 (Radford et al., 2019), have shown great promise in many applications of natural language generation, such as stylized text generation (Syed et al., 2019) and dialog system (Wolf et al., 2019). PLM is obtained by first pre-training on large- scaled raw sentences (always general domain cor- pus), and then used in downstream tasks by fine- tuning on task-specific datasets (always from some specific domains). Specifically, given a pre-trained GPT-2 model, to generate sentences of email do- main, we always need to fine-tune the GPT-2 on a small set of email domain corpus. However, we argue that to get desired sentence outputs, fine-tuning PLM on a specific domain dataset is not necessarily the best, especially when the fine-tuning dataset is of a small size. Typically, fine-tuning is conducted through Maximum Like- lihood Estimation (MLE), with which the result- ing model distribution will be asymptotically con- sistent with true distribution when the fine-tuning dataset has infinite data samples. But it is not the 1 Page . 图一 p𝑇rue(𝑥) p𝑀odel(𝑥) Over-estimation Under-estimation a c d b Figure 1: The over- and under-estimation problems of the model distribution. For example, sample b repre- sents the simple sentence “Yes .”, whose probability is over-estimated. Its model NLL (4.01, negative log- likelihood) is significantly lower than the 95% confi- dence interval of its real NLL [4.89, 5.37], which is estimated on the training set. case of fine-tuning on small datasets, which al- ways leads to the mismatch problem of the real and model distributions. Specifically, MLE minimizes the Kull- back–Leibler (KL) divergence between model and true distributions. Theis et al. (2016) point out that minimizing KL avoids assigning an extremely small probability to any data point but assigns a lot of probability mass to non-data regions, which leads to a gap between PReal and PModel. Additionally, simple data patterns in the fine-tuning dataset could be easily memorized and over-estimated. Meanwhile, the complex ones may be under-estimated. The above problem is not severe with adequate data samples, but non-trivial when the size of the fine-tuning dataset …

Content truncated to 500 words