Active Learning for Coreference Resolution

2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 508–512, Montr´eal, Canada, June 3-8, 2012. c⃝2012 Association for Computational Linguistics Active Learning for Coreference Resolution Florian Laws1 Florian Heimerl2 Hinrich Sch¨utze1 1 Institute for Natural Language Processing (IMS) Universit¨at Stuttgart florian.laws@ims.uni-stuttgart.de 2 Institute for Visualization and Interactive Systems Universit¨at Stuttgart florian.heimerl@vis.uni-stuttgart.de Abstract We present an active learning method for coreference resolution that is novel in three re- spects. (i) It uses bootstrapped neighborhood pooling, which ensures a class-balanced pool even though gold labels are not available. (ii) It employs neighborhood selection, a selection strategy that ensures coverage of both posi- tive and negative links for selected markables. (iii) It is based on a query-by-committee selec- tion strategy in contrast to earlier uncertainty sampling work. Experiments show that this new method outperforms random sampling in terms of both annotation effort and peak per- formance. 1 Introduction Coreference resolution (CR) – the task of determin- ing if two expressions in natural language text re- fer to the same real-world entity – is an important NLP task. One popular approach to CR is super- vised classification. This approach needs manually labeled training data that is expensive to create. Ac- tive learning (AL) is a technique that can reduce this cost by setting up an interactive training/annotation loop that selects and annotates training examples that are maximally useful for the classifier that is being trained. However, while AL has been proven successful for many other NLP tasks, such as part- of-speech tagging (Ringger et al., 2007), parsing (Osborne and Baldridge, 2004), text classification (Tong and Koller, 2002) and named entity recogni- tion (Tomanek et al., 2007), AL has not been suc- cessfully applied to coreference resolution so far. In this paper, we present a novel approach to AL for CR based on query-by-committee sampling and bootstrapping and show that it performs better than a number of baselines. 2 Related work Coreference resolution. The perhaps most widely used supervised learning approach to CR is the mention-pair model (Soon et al., 2001). This model classifies links (pairs of two mentions) as corefer- ent or disreferent, followed by a clustering stage that partitions entities based on the link decisions. Our AL method is partially based on the class balancing strategy proposed by Soon et al. (2001). While models other than mention-pair have been proposed (Culotta et al., 2007), none performs clearly better as evidenced by recent shared evalu- ations such as SemEval 2010 (Recasens et al., 2010) and CoNLL 2011 (Pradhan et al., 2011). Active learning. The only existing publication on AL for CR that we are aware of is (Gasperin, 2009). She uses a mention-pair model on a biomed- ical corpus. The classifier is Naive Bayes and the AL method uncertainty sampling (Lewis and Gale, 1994). The results are negative: AL is not bet- ter than random sampling. In preliminary experi- ments, we replicated this result for our corpus and our system: Uncertainty sampling is not better than …

Content truncated to 500 words