Semi-Supervised Active Learning for Sequence Labeling

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 1039–1047, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Semi-Supervised Active Learning for Sequence Labeling Katrin Tomanek and Udo Hahn Jena University Language & Information Engineering (JULIE) Lab Friedrich-Schiller-Universit¨at Jena, Germany {katrin.tomanek|udo.hahn}@uni-jena.de Abstract While Active Learning (AL) has already been shown to markedly reduce the anno- tation efforts for many sequence labeling tasks compared to random selection, AL remains unconcerned about the internal structure of the selected sequences (typ- ically, sentences). We propose a semi- supervised AL approach for sequence la- beling where only highly uncertain sub- sequences are presented to human anno- tators, while all others in the selected se- quences are automatically labeled. For the task of entity recognition, our experiments reveal that this approach reduces annota- tion efforts in terms of manually labeled tokens by up to 60 % compared to the stan- dard, fully supervised AL scheme. 1 Introduction Supervised machine learning (ML) approaches are currently the methodological backbone for lots of NLP activities. Despite their success they create a costly follow-up problem, viz. the need for human annotators to supply large amounts of “golden” annotation data on which ML systems can be trained. In most annotation campaigns, the lan- guage material chosen for manual annotation is se- lected randomly from some reference corpus. Active Learning (AL) has recently shaped as a much more efficient alternative for the creation of precious training material. In the AL paradigm, only examples of high training utility are selected for manual annotation in an iterative manner. Dif- ferent approaches to AL have been successfully applied to a wide range of NLP tasks (Engel- son and Dagan, 1996; Ngai and Yarowsky, 2000; Tomanek et al., 2007; Settles and Craven, 2008). When used for sequence labeling tasks such as POS tagging, chunking, or named entity recogni- tion (NER), the examples selected by AL are se- quences of text, typically sentences. Approaches to AL for sequence labeling are usually uncon- cerned about the internal structure of the selected sequences. Although a high overall training util- ity might be attributed to a sequence as a whole, the subsequences it is composed of tend to ex- hibit different degrees of training utility. In the NER scenario, e.g., large portions of the text do not contain any target entity mention at all. To further exploit this observation for annotation pur- poses, we here propose an approach to AL where human annotators are required to label only uncer- tain subsequences within the selected sentences, while the remaining subsequences are labeled au- tomatically based on the model available from the previous AL iteration round. The hardness of sub- sequences is characterized by the classifier’s con- fidence in the predicted labels. Accordingly, our approach is a combination of AL and self-training to which we will refer as semi-supervised Active Learning (SeSAL) for sequence labeling. While self-training and other bootstrapping ap- proaches often fail to produce good results on NLP tasks due …

Content truncated to 500 words