Subsequence Based Deep Active Learning for Named Entity Recognition

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4310–4321 August 1–6, 2021. ©2021 Association for Computational Linguistics 4310 Subsequence Based Deep Active Learning for Named Entity Recognition Puria Radmard1,2,3 pr450@cam.ac.uk Yassir Fathullah2 yf286@cam.ac.uk 1University College London 2University of Cambridge 3Vector AI Aldo Lipani1,3 aldo.lipani@ucl.ac.uk Abstract Active Learning (AL) has been successfully applied to Deep Learning in order to drasti- cally reduce the amount of data required to achieve high performance. Previous works have shown that lightweight architectures for Named Entity Recognition (NER) can achieve optimal performance with only 25% of the original training data. However, these meth- ods do not exploit the sequential nature of language and the heterogeneity of uncertainty within each instance, requiring the labelling of whole sentences. Additionally, this standard method requires that the annotator has access to the full sentence when labelling. In this work, we overcome these limitations by allow- ing the AL algorithm to query subsequences within sentences, and propagate their labels to other sentences. We achieve highly efficient results on OntoNotes 5.0, only requiring 13% of the original training data, and CoNLL 2003, requiring only 27%. This is an improvement of 39% and 37% compared to querying full sen- tences. 1 Introduction The availability of large datasets has been key to the success of deep learning in Natural Language Pro- cessing (NLP). This has galvanized the creation of larger datasets in order to train larger deep learning models. However, creating high quality datasets is expensive due to the sparsity of natural language, our inability to label it efficiently compared to other forms of data, and the amount of prior knowledge required to solve certain annotation tasks. Such a problem has motivated the development of new Active Learning (AL) strategies which aim to effi- ciently train models, by automatically identifying the best training examples from large amounts of Code is made available on: https://github.com/ puria-radmard/RFL-SBDALNER unlabeled data (Wei et al., 2015; Wang et al., 2017; Tong and Koller, 2002). This tremendously reduces human annotation effort as much fewer instances need to be labeled manually. To minimise the amount of data needed to train a model, AL algorithms iterate between training a model, and querying information rich instances to human annotators from a pool of unlabelled data (Huang et al., 2014). This has been shown to work well when the queries are ‘atomic’—a single an- notation requires a unit labour, and describes en- tirely the instance to be annotated. Conversely, each instance of structured data, such as sequences, require multiple annotations. Hence, such query se- lection methods can result in a waste of annotation budget (Settles, 2011). For example, in Named Entity Recognition (NER), each sentence is usually considered an in- stance. However, because each token has a sepa- rate label, annotation budgeting is typically done on a token basis (Shen et al., 2017). Budget wast- ing may therefore arise from the heterogeneity of uncertainty across each sentence; a sentence …

Content truncated to 500 words