Selecting Informative Contexts Improves Language Model Fine-tuning

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1072–1085 August 1–6, 2021. ©2021 Association for Computational Linguistics 1072 Selecting Informative Contexts Improves Language Model Fine-tuning Richard Antonello UT Austin rjantonello@utexas.edu Nicole M. Beckage Intel Labs nicole.beckage@intel.com Javier S. Turek Intel Labs javier.turek@intel.com Alexander G. Huth UT Austin huth@cs.utexas.edu Abstract Language model fine-tuning is essential for modern natural language processing, but is computationally expensive and time- consuming. Further, the effectiveness of fine- tuning is limited by the inclusion of training examples that negatively affect performance. Here we present a general fine-tuning method that we call information gain filtration for im- proving the overall training efficiency and final performance of language model fine-tuning. We define the information gain of an exam- ple as the improvement on a validation met- ric after training on that example. A sec- ondary learner is then trained to approximate this quantity. During fine-tuning, this learner selects informative examples and skips unin- formative ones. We show that our method has consistent improvement across datasets, fine- tuning tasks, and language model architec- tures. For example, we achieve a median per- plexity of 54.0 on a books dataset compared to 57.3 for standard fine-tuning. We present sta- tistical evidence that offers insight into the im- provements of our method over standard fine- tuning. The generality of our method leads us to propose a new paradigm for language model fine-tuning — we encourage researchers to re- lease pretrained secondary learners on com- mon corpora to promote efficient and effec- tive fine-tuning, thereby improving the perfor- mance and reducing the overall energy foot- print of language model fine-tuning. 1 Introduction Language modeling is the task of generating lan- guage from context. This is often framed as an autoregressive task, where a model predicts the conditional probability of the next word based on the sequence of previously observed or generated tokens. Language modeling has seen a recent surge in relevance thanks to its success as a pre- training objective for self-supervised representa- tion learning. The most prominent language mod- els today are Transformer-based models (Vaswani et al., 2017) such as BERT (Devlin et al., 2019) and GPT-2 (Radford et al., 2019). Language models are most commonly trained with backpropagation using traditional NLP loss functions such as cross entropy. These loss func- tions are designed so that the models are rewarded for assigning high probability to text that appears commonly in the training corpus. The energy and computational costs of training a state-of-the-art language model from scratch are very high, to the point of impracticality for most researchers. One recent estimate suggests that training a sin- gle state-of-the-art model with architecture search takes more energy than five cars will use in their entire lifetimes (Strubell et al., 2019). In prac- tice, this cost is sidestepped by pretraining, where a language model is trained once and then released publicly. This language model can then be up- dated for use in …

Content truncated to 500 words