Cold-start Active Learning through Self-supervised Language Modeling

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7935–7948, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7935 Cold-start Active Learning through Self-supervised Language Modeling Michelle Yuan∗ University of Maryland myuan@cs.umd.edu Hsuan-Tien Lin National Taiwan University htlin@csie.ntu.edu.tw Jordan Boyd-Graber University of Maryland jbg@umiacs.umd.edu Abstract Active learning strives to reduce annotation costs by choosing the most critical examples to label. Typically, the active learning strat- egy is contingent on the classification model. For instance, uncertainty sampling depends on poorly calibrated model confidence scores. In the cold-start setting, active learning is impractical because of model instability and data scarcity. Fortunately, modern NLP pro- vides an additional source of information: pre- trained language models. The pre-training loss can find examples that surprise the model and should be labeled for efficient fine-tuning. Therefore, we treat the language modeling loss as a proxy for classification uncertainty. With BERT, we develop a simple strategy based on the masked language modeling loss that minimizes labeling costs for text classification. Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and computation time. 1 Introduction Labeling data is a fundamental bottleneck in ma- chine learning, especially for NLP, due to annota- tion cost and time. The goal of active learning (AL) is to recognize the most relevant examples and then query labels from an oracle. For instance, policy- makers and physicians want to quickly fine-tune a text classifier to understand emerging medical conditions (Voorhees et al., 2020). Finding labeled data for medical text is challenging because of pri- vacy issues or shortage in expertise (Dernoncourt and Lee, 2017). Using AL, they can query labels for a small subset of the most relevant documents and immediately train a robust model. Modern transformer models dominate the leader- boards for several NLP tasks (Devlin et al., 2019; Yang et al., 2019). Yet the price of adopting ∗⋆Work done while visiting National Taiwan University. transformer-based models is to use more data. If these models are not fine-tuned on enough exam- ples, their accuracy drastically varies across differ- ent hyperparameter configurations (Dodge et al., 2020). Moreover, computational resources are a major drawback as training one model can cost thousands of dollars in cloud computing and hun- dreds of pounds in carbon emissions (Strubell et al., 2019). These problems motivate further work in AL to conserve resources. Another issue is that traditional AL algorithms, like uncertainty sampling (Lewis and Gale, 1994), falter on deep models. These strategies use model confidence scores, but neural networks are poorly calibrated (Guo et al., 2017). High confidence scores do not imply high correctness likelihood, so the sampled examples are not the most uncertain ones (Zhang et al., 2017). Plus, these strategies sample one document on each iteration. The single- document sampling requires training the model after each query and increases the overall expense. These limitations of modern NLP models illus- trate a twofold effect: they show a greater need for AL and make AL more difficult to deploy. Ideally, AL …

Content truncated to 500 words