Learning How to Active Learn by Dreaming

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4091–4101 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4091 Learning How to Active Learn by Dreaming Thuy-Trang Vu Faculty of Information Technology Monash University, Australia trang.vuthithuy@monash.edu Ming Liu School of Info Technology Deakin University, Australia m.liu@deakin.edu Dinh Phung Faculty of Information Technology Monash University, Australia dinh.phung@monash.edu Gholamreza Haffari Faculty of Information Technology Monash University, Australia gholamreza.haffari@monash.edu Abstract Heuristic-based active learning (AL) methods are limited when the data distribution of the underlying learning problems vary. Recent data-driven AL policy learning methods are also restricted to learn from closely related do- mains. We introduce a new sample-efficient method that learns the AL policy directly on the target domain of interest by using wake and dream cycles. Our approach interleaves between querying the annotation of the se- lected datapoints to update the underlying stu- dent learner and improving AL policy using simulation where the current student learner acts as an imperfect annotator. We evaluate our method on cross-domain and cross-lingual text classification and named entity recogni- tion tasks. Experimental results show that our dream-based AL policy training strategy is more effective than applying the pretrained policy without further fine-tuning, and better than the existing strong baseline methods that use heuristics or reinforcement learning. 1 Introduction Obtaining adequate annotated data is often expen- sive and time consuming for many real-world NLP tasks. Active learning (AL) aims to economically learn an accurate model by reducing the annota- tion cost. It is based on the premise that a model can get better performance if it is allowed to pre- pare its own training data, by choosing the most beneficial data points and querying their annota- tions from annotators. For example, the learner can identify its knowledge gaps in order to select the most informative query data points. The core AL problem is how to identify the most beneficial query data points. Tradition- ally, they are identified using various hand crafted heuristics (Settles, 2012). Recent work has investi- gated learning the AL query strategy from the data (Fang et al., 2017; Bachman et al., 2017; Wood- ward and Finn, 2017; Contardo et al., 2017; Liu et al., 2018a; Pang et al., 2018), as engineered heuristics are not flexible to exploit characteristics inherent to a given problem. These works are all based on the idea that aims to learn an AL query strategy on a related problem for which enough annotated data exist via AL simulations, and then transfers it to the target AL scenario of interest. The success of this approach, however, highly de- pends on the relatedness of the source and target AL problems, as the transferred AL strategy is not adapted to the characteristics of the target AL problem. To address this mismatch challenge, we intro- duce a new approach that learns an AL query strategy directly for the target problem of interest. Starting from an initial (pre-trained) AL strategy, our approach interleaves between querying the …

Content truncated to 500 words