Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1982–1995 June 6–11, 2021. ©2021 Association for Computational Linguistics 1982 Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation Rishi Hazra, Parag Dutta, Shubham Gupta, Mohammed Abdul Qaathir, Ambedkar Dukkipati {rishihazra, paragdutta, shubhamg, mohammedq, ambedkar}@iisc.ac.in Indian Institute of Science, Bangalore Abstract While deep learning is a powerful tool for natu- ral language processing (NLP) problems, suc- cessful solutions to these problems rely heav- ily on large amounts of annotated samples. However, manually annotating data is expen- sive and time-consuming. Active Learning (AL) strategies reduce the need for huge vol- umes of labeled data by iteratively selecting a small number of examples for manual annota- tion based on their estimated utility in train- ing the given model. In this paper, we argue that since AL strategies choose examples in- dependently, they may potentially select simi- lar examples, all of which may not contribute significantly to the learning process. Our pro- posed approach, Active2 Learning (A2L), ac- tively adapts to the deep learning model being trained to eliminate such redundant examples chosen by an AL strategy. We show that A2L is widely applicable by using it in conjunction with several different AL strategies and NLP tasks. We empirically demonstrate that the pro- posed approach is further able to reduce the data requirements of state-of-the-art AL strate- gies by ≈ 3 − 25% on an absolute scale on multiple NLP tasks while achieving the same performance with virtually no additional com- putation overhead. 1 Introduction Active Learning (AL) (Freund et al., 1997; McCal- lum and Nigam, 1998) reduces the need for large quantities of labeled data by intelligently select- ing unlabeled examples for expert annotation in an iterative process. Many Natural Language Pro- cessing (NLP) tasks like sequence tagging (NER, POS) and Neural Machine Translation (NMT) are very data-intensive and require a meticulous, time- consuming, and costly annotation process. On the other hand, unlabeled data is practically unlimited. Due to this, many researchers have explored appli- cations of active learning for NLP (Thompson et al., 1999; Figueroa et al., 2012). A general AL method proceeds as follows: (i) The partially trained model for a given task is used to (possibly incorrectly) annotate the unlabeled examples. (ii) An active learning strategy selects a subset of the newly la- beled examples via a criterion that quantifies the perceived utility of examples in training the model. (iii) The experts verify/improve the annotations for the selected examples. (iv) These examples are added to the training set, and the process repeats. AL strategies differ in the criterion used in step (ii). We claim that all AL strategies select redundant examples in step (ii). If one example satisfies the se- lection criterion, then many other similar examples will also satisfy it (see the next paragraph for de- tails). As the examples are selected independently, AL strategies redundantly choose all of these exam- ples …

Content truncated to 500 words