On Curriculum Learning for Commonsense Reasoning

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 983 - 992 July 10-15, 2022 ©2022 Association for Computational Linguistics On Curriculum Learning for Commonsense Reasoning Adyasha Maharana Mohit Bansal Department of Computer Science University of North Carolina at Chapel Hill {adyasha, mbansal}@cs.unc.edu Abstract Commonsense reasoning tasks follow a stan- dard paradigm of finetuning pretrained lan- guage models on the target task data, where samples are introduced to the model in a ran- dom order during training. However, recent research suggests that data order can have a significant impact on the performance of fine- tuned models for natural language understand- ing. Hence, we examine the effect of a human- like easy-to-difficult curriculum during finetun- ing of language models for commonsense rea- soning tasks. We use paced curriculum learning to rank data and sample training mini-batches with increasing levels of difficulty from the ranked dataset during finetuning. Further, we investigate the effect of an adaptive curriculum, i.e., the data ranking is dynamically updated during training based on the current state of the learner model. We use a teacher model to measure difficulty of each sample and exper- iment with three measures based on question answering probability, variability and out-of- distribution. To understand the effectiveness of curriculum learning in various scenarios, we apply it on full model fine-tuning as well as parameter-efficient prompt-tuning settings. Our results show that fixed as well as adap- tive curriculum learning significantly improve performance for five commonsense reasoning tasks, i.e., SocialIQA, CosmosQA, CODAH, HellaSwag, WinoGrande in both tuning set- tings. Further, we find that prioritizing the diffi- cult samples in the tail end of training improves generalization to unseen in-domain data as well as out-of-domain data. Our work provides evi- dence and encourages research into curriculum learning for commonsense reasoning.1. 1 Introduction Curriculum learning (Elman, 1993; Bengio et al., 2009) is an alternative to the typical uniform ran- dom sampling of training data and is motivated by 1Code: https://github.com/adymaharana/ curriculum_learning Teacher Model Original Dataset Ranked Dataset (Curriculum) Learner Model Paced Learning Adaptive Curriculum Learning learner state Figure 1: Curriculum Learning. The curriculum of the learner model i.e., the ranked dataset for finetuning is prepared using scores from the teacher model. The ranking remains unchanged in fixed curriculum learning whereas the learner state is used as feedback to update the ranking in adaptive curriculum learning. the gradual progression of human learning from easier to difficult concepts (see Figure 1). In the machine learning paradigm, a ‘teacher’ ranks the training samples from easy to difficult and intro- duces them to the ‘learner’ in that order. Dodge et al. (2020) show that randomly initialized train- ing data orders can lead to large variance in model performance on the GLUE benchmark (Wang et al., 2018). In light of such evidence, we seek to answer: Does a meaningful data order such as a curriculum based on model confidence or dataset distribution outperform a random data order? Such experiments have been carried …

Content truncated to 500 words