Dynamic Data Selection for Curriculum Learning via Ability Estimation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 545–555 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 545 Dynamic Data Selection for Curriculum Learning via Ability Estimation John P. Lalor1∗, Hong Yu2,3 1Department of IT, Analytics, and Operations, University of Notre Dame 2 Department of Computer Science, University of Massachusetts Lowell 3 College of Information and Computer Sciences, University of Massachusetts Amherst john.lalor@nd.edu, hongyu@cs.umass.edu Abstract Curriculum learning methods typically rely on heuristics to estimate the difficulty of training examples or the ability of the model. In this work, we propose replacing difficulty heuris- tics with learned difficulty parameters. We also propose Dynamic Data selection for Cur- riculum Learning via Ability Estimation (DDa- CLAE), a strategy that probes model ability at each training epoch to select the best training examples at that point. We show that models using learned difficulty and/or ability outper- form heuristic-based curriculum learning mod- els on the GLUE classification tasks. 1 Introduction Curriculum learning trains a model by using easy examples first and gradually adding more difficult examples. It can speed up learning and improve generalization in supervised learning models (Ben- gio et al., 2009; Amiri et al., 2017; Platanios et al., 2019). A major drawback of existing curriculum learning techniques is that they rely on heuristics to measure the difficulty of data, and either ignore the competency of the model during training or rely on heuristics there as well. For example, sentence length is often used as a proxy for difficulty in NLP tasks (Bengio et al., 2009; Platanios et al., 2019). Such heuristics can be useful but have limitations. First, the heuristic chosen may not actually be a proxy for difficulty. Depending on the task, long sequences could signal easier or harder examples, or have no signal for difficulty. Second, a model’s notion of difficulty may not align with the heuristic imposed by a human developing the model. It may be that examples that appear difficult for the human are in fact easy for the model to learn. Competency was recently introduced as a mech- anism to determine when new examples should be ∗Work performed while at UMass Amherst. added to the training data (Platanios et al., 2019). However, in that work competency is a monotoni- cally increasing function of a pre-determined initial value. Once set, competency is not evaluated dur- ing training. Ideally, model competency should be measured at each training epoch, so that the train- ing data could be appropriately matched with the model at a given point in the training. If a model is improving, then more difficult training data can be added at the next epoch. But if performance declines, then those difficult examples can be re- moved, and a smaller, easier training set can be used in the next epoch. In this study, we propose to estimate both the difficulty of examples and the ability of deep learn- ing models as latent variables based on model per- formance using Item Response Theory (IRT), a well-studied methodology …

Content truncated to 500 words